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ABSTRACT 

This report reviews and synthesizes the findings of 
the survey and fieldwork as presented in Volumes II and III. In 
particular, it summarizes the evidence concerning the effects of 
federal change agent polidy. Section 1 introduces the report. Section 
2 describes the theoretical approach that served as the basis for 
data collection and analysis. Sections 3 and 4 # respectively, present 
findings about the innovative process and about the factors affecting 
implementation and continuation. These sections do not present the 
actual analyses but use footnotes to refer to evidence presented in 
the ot,her volumes. Section 5 goes beyond immediate data to raise 
questions about the policy instruments used in federal change agent 
programs. These policy implications are tentative for several 
reasons. First, this report presents only the findings of the first 
year of a two-phase study. Second, this is exploratory research — in 
effect the work consists of hypotheses that require more refined 
testing. Nonetheless, since policy can seldom afford the slow pace of 
scientific confirmation, conclusions supported by both the evidence 
and experience are offered. (Author) 
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PREFACE 



Rand is conducting, under the sponsorship t)f the U.S. Office of Education, a 
several-year study of federally funded programs designed to introduce and spread 
innovative practices in public schools. These change agent programs normally offer 
temporary federal funding to school districts as "seed money." If an innovation is 
successful, it is assumed that the district will continue and disseminate part or all 
of the project using other sources of funds. The Rand study examines four such 
federal change agent programs— Elementary and Secondary Education Act Title III, 
Innovative Projects; Elementary and Secondary Education Act Title VII, Bilingual 
Projects; Vocational Education Act, 1968 Amendments, Part D, Exemplary Pro- 
grams; and the Right-To-Read Program. The study identifies what tends to promote 
various kinds of changes in the schools and what doesn't; in particular, the Rand 
study will identify for federal, state, and local policymakers the^nature, permanence, 
and extent of dissemination of innovations that are associated with the various 
federal programs and with various federal, state, and local practices. 

A series of five reports describes the results of the first year of the Rand study 
(July 1973 to July 1974). Volume I (R-1589/1-HEW, A Model of Educational Change) 
provides a theoretical perspective for the Rand study by analyzing the current state 
of knowledge of planned change in education and by proposing a conceptual model 
of factors affecting change processes within school districts. 1 

Volume II (R-1589/2-HEW, Factors Affecting Change Agent Projects) contains 
the analysis of survey data collected from a national sample of 293 projects in 18 
states during November and December 1973. 

Volume III (R-1589/3-HEW, The Process of Change) summarizes the results of 
29 case studies of change agent projects conducted by Rand staff members and 
consultants in 25* school districts during April and May 1974. These case studies 
were chosen from the original sample of 293 projects initially surveyed. Volume III 
also describes the role of state education agencies in choosing and disseminating the 
change agent projects. 

Four technical appendixes to Vol. Ill describe in detail the federal program 
management approach, state education agency participation, and case studies for 
each of the programs in the study: Title III, App. A; Reading, App. B; Bilingual 
Education, App. C; and Career Education, App. D. Appendix A should be of particu- 
lar interest to researchers or practitioners concerned with the introduction of new 
approaches to classroom instruction. 

This report, Vol. IV of the series, summarizes the findings of Vois. I, II, and III, 
and also synthesizes extensive data collected by Rand on federal-level program 
strategy and management for each of the* change agent programs. Volume IV also 
includes a discussion of alternative federal strategies for promoting innovation. 

Volume V (R-1589/5-HEW, Executive Summary) presents a distillation of the 
study's methods and results for a general audience. 

Subsequent research will collect additional data on Titles III and VII of ESEA, 
with particular focus on projects whose federal funding has expired. 

' Because of Rand's interest in advancing knowledge of organizational behavior in educational insti 
tutions. the research underlying this report was supported in part by an allocation of Rand corporate 
research funds. ^ 
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SUMMARY 



.BACKGROUND 

This is a report on the first year (July 1973 to July 1974) of Rand's work in a 
study of four federal programs (ESEA Title III; ESEA Title VII; Vocational Educa- 
tion, 1968 Amendments, Part D; and Right-To-Read) aimed at promoting education- 
al change in the public schools by paying for the costs of innovative projects for a 
trial period. This study, commissioned by the U.S. Office of Education, aims to help^ 
improve the ways that policies are made and carried out by describing how the 
process of innovation operates for these projects and by trying to account for the „ 
factors that affect their outcomes. The second phase of the work, covering the 
1974-75 and 1975-76 school years, will investigate how innovative projects are con- 
tinued and spread after federal support ends. 

The four programs rest on common assumptions, but each has its own focus and 
management strategy. The common assumptions are: 

• American education should be doing fetter with respect to a variety of 
goals. . 

• Educational practices can be improved within the existing educational 
structure. / 

• Change can be introduced and sustained by providing "seed money" to 
* some districts to encourage innovations that, if successful, will be con- 
tinued in the original sites and will be adopted selectively by other schools 
and districts. 

Each of the programs is also distinct: 

• Title III (funded at approximately $150 million annually in recent years) 
has the broadest aims. It is designed to encourage educational improve- 
ment at the school and district' level by introducing new practices and by 
spreading existing model practices to districts that are not aware of them. 
In recent years 85 percent of the funding has been state-administered, and 
the remaining 15 percent directly administered by USOE. 

• Right-To-Read ($12 million annually) seeks to create a national education- 
al priority for reading, particularly among disadvantaged students. This 
study focuses on one aspect of the program, administered by USOE— 
demonstration projects at the school level. 

• Vocational Education, Part D ($16 million annually) was designed to cre- 
ate exemplary programs that would enhance career awareness and readi- 
ness. Half of the funds are administered directly by USOE and the balance 
by the states. 

• Title VII Bilingual Education ($45-$85 million annually) is aimed at pro- 
viding model projects for children of limited English-speaking ability, ar d 
also to maintain and encourage "cultural pluralism" in American educa- 
tion, with strong political support from many people of Spanish-language 
origin. ir 
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THE-RANp STUDY 

In light of numerous findings that have raised questions ibout the effectiveness 
and transferability of educational innovation, USOE asked Rand to focus on four 
questions in the change agent study: 

• How should the nature and extent of innovation and dissemination of 
change in the public schools be assessed? 

• How do schoOl districts select, introduce, implement, incorporate, and 
spread different kinds of innovations? 

• How do differences in the federal programs, in project characteristics, and 
in local settings affect how projects are begun, carried out, continued on 
local funds, and disseminated? 

• What should federal policies be toward educational innovation in light of 
the political, financial, and organizational constraints that the federal gov- 
ernment faces in its dealings with the public schools? 

The research design for the first year of the study approached' these issues 
through four major research tasks: 

1. An extensive review of the literature on educational innovations, leading 
to development of a theoretical approach toward the subject that serves as 
the basis for data collection and analysis. 

2. A nationwide survey in 18* states of 293 change agent projects, each in its 
last or next to last year of federal funding. The survey, conducted for Rand 
by the National Opinion Research Center in December 1973 and January 
1974, included personal interviews with 1735 people at all levels in the 
school district, from superintendent to classroom teacher. It sought to find 
out what factors influence the outcomes of change agent projects. These 
survey data, supplemented by school district data, were analyzed using 
such statistical procedures as multiple regression and factor analysis. 

3. Field studies conducted in April and May 1973 by Rand staff at 29 projects, 
drawn from the survey sample. The staff observed the projects in operation 
in classrooms and schools and interviewed project participants and district 
officials in order to understand how the innovative process worked. The 
field work sample was drawn so as to provide a comparison of similar inno- 
vations operating in different local settings and supported by different 
federal programs. In particular, the fieldwork cases included classroom 
organization, staff development, reading, bilingual, and career education 
projects. 

4. Rand staff interviews with federal and SEA officials who work on the four 
change agent programs. These included telephone interviews with 54 SEA 
officials in 18 states, visits to 9 SEAs for more detailed personal interviews, 
and a scries of personal interviews with officials at USOE and the Depart- 
ment of Health, Education, and Welfare (HEW). 

In the final phase of the work (November 1974 to September 1976), Rand will 
examine what happens to projects in the two largest change agent programs when 
federal funding stops. The work will again be based on surveys and field studies and 
will include Title III and Title VII projects that were visited during 1973-74. This 
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phase of the work will allow us to test the first-year findings and will also allow us 
to test hypotheses about continuation and dissemination that were not explored 
during the first year. 

THE RESEARCH APPROACH * - ^ 

As part of its first major task, Rand reviewed the literature on educational 
innovations and found that the past decade of federal efforts to stimulate change in 
local school systems has led to disappointing results. This apparent disappointment 
may be due less to inadequate educational technologies or treatments than to the 
way these promising treatments were implemented in the local institutional setting. 
We concluded that an essential issue for policymakers promoting change was to 
develop a systematic understanding of implementation. 

Therefore, to guide the research, we focused on: (1) developing a model of the 
innovative process that centered on implementation, (2) defining outcome measures 
that assess the effectiveness of implementation and the extent to which the change 
agent projects meet the policy goals of project continuation and dissemination, and 
(3) identifying factors affecting the innovative process and, consequently, the out- 
comes of innovative projects: 

The Model of the Innovative Process 

The model hypothesizea three stages in the life of an innovative project: 

• Initiation, when LEA officials plan projects and decide which ones to sup- 
port. 

• Implementation, when the project confronts the reality of the institutional 
setting and project plans must be translated into practice. We hypothesize 
that effective implementation requires mutual adaptation between the 
project as planned^and the institutional setting, in which each must adjust 
to the demands of the other. 

• Incorporation, when the innovative practice loses its "special project" 
status and becomes part of the routinized behavior of the district. In this 
phase the project may be continued in whole or in part as a result of 
deliberate district decisions, or aspects of the innovation may be incorpo- 
rated by individual teachers with or without formal district support. 

Defining Outcomes 

Because innovative projects must be implemented before they can affect stu- 
denteand because they are seldom implemented as planned, we defined project 
"outcjeanes" that measured the effectiveness of implementation: 

• Perceived success: the, relative extent to which project participants be- 
lieved that goals were achieved. 

• Change in behavior: the type and extent of change in teacher and admin- 
istrator behavior as perceived by participants. 

• Fidelity of implementation: the extent to which the project was imple- 
mented as originally planned. *y 
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We used another policy-relevant project "outcome" involving the incorporation 
stage: 

• (Expected) continuation: the extent to which the LEA continued project 
activities after federal funds were withdrawn. 1 

Factors Affecting Implementation and Continuation 

We hypothesized that project outcomes are largely determined by the interplay 
among characteristics of the project, the institutional setting, and federal policies. 
We developed measures for each of these factors, which were used to determine their 
relative effects on project outcomes. 



FINDINGS 

Our findings fall into two categories: those relating to the process of innovation 
and those relating to factors affecting the innovative process and, as a consequence, 
project outcomes. 

The Process of Innovation 

During the initiation stage, the interaction of several factors tended to result 
in initiation processes that could be characterized either as opportunistic (designed 
primarily to take advantage of the availability of external funding, with relatively 
little LEA commitment to project goals) or as problem solving (when the project was 
seen as helping to meet local needs in light of present and expected future realities). 
The motion that characterized opportunism or problem solving had pervasive 
effects on implementation and incorporation. 

Our evidence suggests that the "search for alternatives" traditionally assumed 
to be characteristic of the problem-solving approach to innovation did not occur. In 
developing projects, LEAs used information or treatments that were already known 
to local district personnel. This may mean that LEA staff intuitively feel that the 
success and suitability of an innovation depend primarily on local conditions, a view 
that our evidence supports. Thus, local administrators are likely to be skeptical 
about the reported "success" nf educational methods in other districts and tend to 
rely on the advice of local professionals who have a thorough knowledge of particular 
local conditions. 

The implementation stage was not a simple application of a well-defined tech- 
nology to a well-understood setting. Instead, it implied complex and only partly 
predictable interactions between the project and the setting. During this stage, 
projects that were likely to be implemented effectively were characterized by mutual 
adaptation in which the innovation was modified," and the formal and informal 
organizational relationships among staff and among teachers and students were 
altered. In other cases, implementation did not occur. Instead, projects were "imple- 
mented" in pro forma fashion, or simply broke down and were not implemented at 

1 In the first year of the Rand change agent study, we examined projects in their last years of federal 
funding, and hence were able to measure only exacted continuation. 
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all, or were coopted by project participants so that the project was changed to fit 
«, traditional patterns. 

The type of implementation process — mutual adaptation, cooptation, pro forma 
implementation, or breakdown— that characterized a project depended on three 
considerations: the motivations and circumstances involved in the project's initia- 
tion, its substance and scope of change being attempted, and its implementation 
strategy. Mutual adaptation occurred only in problem-solving projects, notably in 
such projects as open classroom that were highly complex and required considerable 
behavioral change on the part of teachers and administrators. 

Incorporation could take place in different forms, ranging from assimilation of 
new practices by the project stafT, with no formal LEA continuation decision re- 
quired, to a conscious decision by the district to commit resources and support to 
extend the project to all or part of the district. 

At the classroom level, projects that replaced existing practice were more likely 
to be incorporated than those that supplemented the existing curriculum. Incorpora- 
tion was more likely under the following conditions: an emphasis on training 
rather than on the introduction of new technology, training focused on practical 
classroom issues rather than on theoretical concepts, and local development of 
materials rather than reliance on outside consultants. Effects of federal programs 
on expected incorporation were indirect. Because projects with extensive provision 
for training and staff development, such as some Title III classroom reorganization 
and staff development innovations, were most likely to have lasting effects on the 
staff, programs funding these innovations indirectly fostered incorporation. 

At the district level, continuation decisions were based on how LEA officials 
perceived the project— whether it was (1) "successful," f2) affordable, (3) important 
to the district's priorities, and (4) politically acceptable. In the case of opportunistic 
projects, the answer^to the first three points was usually negative, while in the case 
of problem-solving projects the answer to all four was often positive— in effect, the 
pattern of expected continuation tended to follow the pattern evidenced during 
initiation. It is important to note that the superintendents' perception of project 
"success" seemed to reflect attitudes formed during initiation rather than after 
evaluation, which was seldom considered seriously. 
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Factors Affecting Implementation and Continuation 

The following variables were considered to be factors that might affect im- 
plementation and continuation: 

1. Project characteristics (educational treatment, ^resource level, substance 
and scope of proposed change, and implementation strategy), 

2. Institutional setting (organizational climate and characteristics of princi- 
pal actors). 

3. Federal policies. 

The project characteristics having important effects on implementation out- 
comes and continuation were the project's implementation strategies (i.e., the deci- 
sions made as to how the innovation would be implemented) and the substance and 
scope of the proposed organic ational change. Effective implementation strategies 
included on-line planning, practical staff training, local development of materials, 
and a "critical mass" of staff working on the project scfthat the individual innovation 
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did not become too isolated. In respect to substance and scope of change, the most 
important elements were: (1) centrality (the perceived educational priority to the 
LEA); (2) a requirement for change in teacher behavior; (3) comprehensive new 
treatments that were complex in that they required a number of changes by a 
number of actors; and (4) consonance between the values and goals of the project and 
those of the stafT and the district. 

Other project characteristics, such as resource level and type of educational 
treatment or technology, had relatively little effect on project outcomes. 

The institutional setting is a crucially important factor in effective implementa- 
tion. The key elements were high teacher morale, support from the principal and 
from district administrators, and teacher willingness to make extra efforts. These 
conditions made initial adaptation more likely, and tended to occur more frequently 
in elementary schools than in high schools. 

A receptive institutional setting provides explicit, steady support for change 
agent efforts and is a necessary but not sufficient condition for effective implementa- 
tion. Mutual adaptation— which we believe is the key to serious change— requires 
an effective implementation strategy, one that takes advantage of institutional 
support! Indeed, the components of the implementation strategy that we found to 
be most effective— adaptive planning, staff training keyed to the local setting, and 
local material development— were those that enabled the support and commitment 
of administrators and staff to be fully engaged. 

Federal policies primarily affected only the initiation stage by inducing adop- 
tion of innovations in areas of federal concern. But these initial influences did not 
have major effects on those factors in the setting and in the project that mostly 
determine the course and outcomes of the innovation. Consequently, federal policies 
had little influence on effective implementation and outcomes. 

Because the policy common to the federal change agent programs had limited 
influence on the innovative process, each federal program could affect project out- 
comes only at the margin. Within this latitude, the differences in management 
strategies of the programs were related to significant but statistically small effects 
on implementation and project outcomes. 

Conclusion 

Our data show that a receptive institutional setting is a necessary but not 
sufficient condition for effective implementation. An implementation strategy that 
promotes mutual adaptation is critical. 

The main factors affecting innovations w£re the institutional setting, particu- 
larly organizational climate and motivations of participants; the implementation 
strategy employed by local innovators to install the project treatment; and the scope 
of change implied by the project relative to its setting. Neither the technology nor 
the project resources nor the different federal management strategies influenced 
outcomes in major ways. Thus, project outcomes did not depend primarily on "in- 
puts" from outside but on internal factors and local decisions. 



TENTATIVE POLICY IMPLICATIONS 

Our first-year research suggests that the following premises express the reali- 
ties of LEA behavior in the innovative p^^sg: 
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1. Implementation— rather than the adoptiWof a technology, the availabil- 
ity of information about it, or the level of funds committed to it— dominates 
the innovative process and its outcomes. 

2. Effective implementation depends on the receptivity of the institutional 
setting to change. 

3. Effective implementation is characterized by the process of mutual adap- 
tation. 

4. Local school systems vary in their capacity to deal with innovations and 
with the stages of the innovative process. 

The policy implications of these four premises are: 

1. Policy should be concerned with more than the mere adoption of change 
agent projects. Federal change agent policy clearly stimulated the initia- 
tion of special projects, but had little effect on the quality or seriousness 
of the implementation efforts. 

2. The critical significance of the institutional setting should come as no 
surprise to policymakers. If educational technologies are not altered and 
adapted to local conditions, they are ineffective; information about prac- 
tices elsewhere seldom goes beyond the level of simple awareness; federal 
money is used for its intended purpose only if the federal purpose is congru- 
ent with local plans. 

School districts use external inputs, but typically are not influenced by 
them to^Change their commitments, motivations, or concern with innova- 
tion; unless the institution is receptive to change, it is unlikely to be stimu- 
lated by these policy instruments. We believe that policies could be de- 
signed to enhance receptivity to change. 

3. If, given a receptive institutional setting, a project's outcomes depend criti- 
cally on local decisions about how 1 the project will be implemented, federal 
policymakers might consider ways of encouraging mutual adaptation 
strategies, which we believe are the key to effective implementation. 

4. Federal change agent programs generally awarded fixed-term grants re- 
gardless of the school districts' ability to introduce and sustain the particu- 
lar innovations represented in their proposals. Yet we observed similar 
innovations being approached artd installed very differently by school dis- 
tricts according to their capacities to innovate. Rather than making blan- 
ket awards of a fixed number of years, federal change agent policies might 
be keyed to the stages of innovation and might promote the development 
of the school districts' capacity to deal with each stage. 

Generally speaking, there appear to be many possibilities for federal policy to 
affect the innovation process despite its essentially local nature and the autonomy 
of school districts. Each possibility for federal leverage raises problems. This interim 
report offers our preliminary thoughts about new policy directions. But, more impor- 
tant, it tries to provide some information and hypotheses that would help policymak- 
ers balance the possibilities and the problems that ari?e from federal efforts to help 
schools change themselves. 
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L INTRODUCTION 



The Rand change agent study, commissioned by the U.S. Office of Education, 
examines four federally sponsored programs that are trying to promote educational 
change in the public schools by paying for the costs of innovative projects for a trial 
period. By examining the process of innovation and by analyzing those differences 
in federal programs, in project characteristics, and in local settings that affect 
innovation, the study aims to provide basic information for the formulation of 
change agent policy. 

This volume is a review of the findingsof the first year of RandV work (July 1973 
to July 1974). It also present^ some preliminary ideas about what the first year's 
findings may mean for future change agent policy. The second phase of the work, 
covering the 1974-75 and 1975-76 school years, will investigate how innovative 
projects are continued and spread after federal support has ended. 



BACKGROUND OF THE CHANGE AGENT STUDY 

During the 1950s and 1960s, three important initiatives, the National Defense 
Education Act of 1958, the Elementary dnd Secondary Education Act of 1965, and 
the legislative modernization of the federal vocational education program in 1963 
and 1968, defined a new federal role in elementary and secondary education— large- 
scale support of federally mandated programs aimed at specified goals: subsidizing 
special curriculum develop jnt, educating the disadvantaged, training young 
people for careers, broadening access to higher education, and encouraging innova- 
tion in the public schools. 

Promoting innovations, which is the subject of this study, accounts for approxi- 
mately 10 percent of the federal aid to public schools which currently exceeds $3.5 
billion annually. The U.S. Office of Education spends most of these funds through 
a number of avenues, one of which includes the so-called change agent programs 
analyzed in this study: 1 

. Elementary and Secondary Education Act Title III, Innovative 
Projects ($150 million annually) . 4 
Elementary and Secondary Education Act Title VII, BilingUaf 
Projects ($45 million) 
* Vocational Education Act, 1968 Amendments, Part D, Exemplary 
Programs ($16 million) 
Right-To-Read ($12 million) 

Each of the change agent programs has a distinct focus and management strate- 
gy^The largest of the programs considered here. Title HI. is designed to improve the 
quality of public education both by introducing model practices that are new to 

1 Other federal programs, not studied here, also mm at encouraging innovations lor example, cer- 
tain programs for handicapped students, experimental schools, educational voucher demonstrations. 
Follow Through, and elements of the Emergency School Assistance Act 



American education and by spreading existing successful practices to schools that 
are not aware of them. The competition for Title III grants of three-year duration 
is open to almost any kind of project that local schools wish to propose. In 1973-74, 
the tirst year of our study, 15 percent of all Title III money was granted directly to 
local education agencies (LEAs) by the Office of Education, and the remaining funds 
were allocated to state education agencies (SEAs), who in turn made grants to LEAs. 

Other federal change agent programs are more narrowly targeted and have 
more specific funding criteria. Right-To-Read, for example, represents an attempt 
by the Office of Education to create a national educational priority for reading, 
particularly among disadvantaged students. The Right-To-Read demonstration pro- 
jects, which are the program component addressed in this study, fund LEAs to use 
a Right-To-Read planning and management approach to carrying out reforms in 
reading instruction. Vocational Education, Part D, was designed to create exem- 
plary programs that would enhance career awareness and readiness. Congress allot- 
ted half of the Part D, Exemplary Program funds to USOE to fund local projects 
directly;*the other half of the appropriations is allotted to the SEAs. 

Title VII (Bilingual Education) originally sought to provide model projects for 
the special needs of children whose English-speaking ability was limited. The pro- 
gram has subsequently also developed into an effort to maintain and encourage 
"cultural pluralism" in American public education, with strong political support 
from many persons of Spanish-language origin. 

Despite these differences in focus and management strategy, the change agent 
programs have a common purpose: the stimulation and spread of educational inno- 
vations. They also have a common policy instrument: the provision of temporary 
funds (3 to 5 years) which, although small relative to the budget of a school district 
(ranging from grants of $10,000 or less to several hundred thousand dollars per 
year), are intended to fund new educational services, not to support existing prac- 
tice. 

These programs also rest on common assumptions. They all assume more or less 
explicitly that American education should be doing better with respect to a variety 
of goals ranging from specific objectives, such as student reading achievement, to the 
broad concerns of student personality and social development. They also assume 
that educational practices, procedures, and methods can be improved within the 
existing educational structure. Federal change agent policy rests on the idea that 
providing funds to a relatively small number of districts to try innovations will 
demonstrate the value of some of these innovations and thus induce other districts 
to adopt them selectively. 

These federal change agent policies are just a sample of many federal efforts to 
promote change in local educational practices over the past decade. A good deal of 
time, money, and research energies have been spent on evaluating these efforts. 2 
This evaluation research, which can be criticized as invalid because of profound 
measurement problems, points to two general findings: 

« Variations in student outcomes have not been consistently related to 
variations in treatments, once nonschool factors are held constant. 

• "Successful" projects have lacked stability and have not been easy to "ex- 
port" from school to school or district to district. 

IS 

2 Volume I of this study reviews the literature. 
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These disappointing findings have raised serious questions about the usefulness 
of federal efforts to promote innovation in the schools, and, more generally, about 
- the prospects for educational reform. 

STUDY DESIGN 

In light of these findings, and of the federal government's own questions about 
the assumptions underlying change agent programs, USOE asked Rand in June 
1973 to study projects funded by the four programs, 3 focusing on the process of 
innovation and the factors affecting innovation. In particular, we were asked to treat 
the following questions: 

• How should the nature and extent of innovation and dissemination of new 
practices in the public schools be assessed? 

• How do school districts select, introduce, implement, incorporate, and 
spread different kinds of innovations? - 

• How do differences in the federal programs, itf project characteristics, and 
in local settings affect how projects are begun, carried out, continued on 
local funds, and disseminated? 

• What should federal policies be toward educational innovation in light of 
the political, financial, and organizational constraints that the federal gov- 
ernment faces in its dealings with the public schools? 

To examine these questions, we developed a research design for the first year of the 
study, which included literature review, development of theory, data collection, 
analysis of data, and preparation of the first-year reports. The tasks, described fully 
in App. A, were as follows: 

1. An extensive review of the literature on educational innovations, leading 
to development of a theoretical approach toward the subject that served as 
the basis for data collection and analysis. 4 

2. A nationwide survey in 18 states of 293 change agent projects, each in its 
last or next to last year of federal fundiijg. The survey, conducted for Rand 
by the National Opinion Research Center of the University of Chicago in 
December 1973 and January 1974, included personal interviews with 1735 
people at all levels in the school district, from superintendent to classroom 
teacher. It sought to find out what factors influence the outcomes of change 
agent projects. The personal interviews elicited information and opinions 
about factors affecting change agent projects— project characteristics, or- 
ganizational features and personal relationships within the schools and the 
district, and federal program effects. In addition, project participants were 
asked about their perceptions of a variety of project "outcomes." These 
survey data, supplemented by school district data, were analyzed using 
such statistical procedures as multiple regression and factor analysis. 5 

3 In this report, we refer to programs when describing the federal change agent initiatives, for 
example, Right-To-Read. We refer to projects when describing the particular innovation selected by a 
school district. 

4 See Vol. I. 

5 See Vol. II for a description of the survey instruments, the sampling procedure, the nature of the 
Rand sample, and the statistical analyses. 
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3. Field studies conducted in April and May 1973 by Rand staff at 29 projects, 
drawn from the survey sample. The staff observed the projects in operation 
in classrooms andschools and interviewed project participants and district 
officials in order to understand how the innovative process worked. The 
(ieldwork sample was drawn so as to provide a comparison of 1 similar inno- 
vations operating in different local settings and supported by different 
federal programs. In particular the fieldwork cases included classroom 
organization, staff development, reading, bilingual, and career education 
projects. 0 

4. Rand staff interviews with federal and SEA officials who work on the four 
change agent programs. These included telephone interviews with 54 SEA 
officials in 18 states, visits to 9 SEAs for more detailed personal interviews, 
and a series of personal interviews with officials at USOE and the Depart- 
ment of Health, Education, and Welfare (HEW). 7 

j 

CONTENTS OF THIS REPORT 

This report reviews and synthesizes the findings of the survey and fieldwork as 
presented in Vols. II and III, respectively. In particular, it summarizes the evidence^ 
concerning the effects of federal change agent policy. Appendix B describes the 
Jiverse management strategies and different priorities of the federal programs. 
Section IV compares these programs in terms of their effects on project outcomes. 
But our primary interest is to use these comparisons to cast light on the effectiveness 
of the policy instruments underlying the change agent programs. Indeed, we are 
specifically not evaluating the management of these programs, nor are we dealing 
with the larger social and political goals for which they were instituted. We focus 
on the innovative process at the local level and ask how these federal programs 
affected the local projects. 

Section II describes the theoretical approach that served as the basis for data 
collection and analysis. Sections III and IV, respectively, present findings about the 
innovative process and about the factors affecting implementation and continuation. 
Because this report is a summary of other volumes of the Rand change agent study, 
these sections do not present the actual analyses but use footnotes to refer to evi- 
dence presented in the other volumes. 

Section V goes beyond our immediate data to raise questions about the policy 
instruments used in federal change agent programs. These policy implications are 
tentative for several reasons. First, this report presents only the findings of the first 
year of a two-phase study. Our final report will have the benefit of one more year 
of research, as well as comments and criticism from federal, state, and local policy- 
makers. Second, this is exploratory research— in effect our work consists of hypothe- 
ses that require more refined testing. Nonetheless, since policy can seldom afford the 
slow pace of scientific confirmation, we do offer conclusions that both the evidence 
and our experience support. 



• See Vol. Ill fonan analysis of the fieldwork data and its appendixes for the case studies. 
7 The appendixes to Vol III contain brief summaries of this work. 



IL THE THEORETICAL APPROACH 



The literature on educational evaluations generally agrees that federal efforts 
to promote innovations have resulted in little consistent or stable improvement in 
student outcomes. The debate about the reasons for this apparent failure centers on 
four explanations: 

1. Schools are already having the maximum possible effect; new practices, 
then, cannot be expected to make a difference. 

2. Innovative ideas and technologies tried thus far are inadequate or under- 
..developed; more R&D is needed. * 

3. Change in student outcomes has occurred, but the measurement and 
analyses are inappropriate or insensitive. 

1 4. Innovative practices are not implemented as planned. 

In our view, the first and second points cannot be judged because, as the third 
explanation maintains, evaluations of innovations are beset with conceptual and 
methW&bgical problems of knowing what to measure and of measuring it validly. 
However, these research difficulties only confound the fundamental problem, which 
is suggested by the fourth explanation: The bridge between a promising idea and its 
impact cn students is implementation, but innovations are seldom implemented as 
planned. 

Thus, innovations may result in disappointing outcomes not because of 
inadequacies of the innovative idea but because of the difficult and uncertain process 
of implementing innovative efforts in an educational system that resists change. 

Moreover, implementation involves complex organizational processes that may 
result in significant modifications in the planned innovation. Experience suggests 
that innovative projects mutate during implementation— that is, they change over 
time within sites— and, moreover, that they display considerable variability from 
one institutional setting to another. 1 Thus, the adoption of an innovation cannot be 
assumed to provfde an accurate forecast of its actual use. In this situation, it may 
be misleading and of little help to policymakers to examine the relationship between 
treatment and student outcomes.without first having a systematic understanding of 
implementation. 

Based on these considerations, we concluded that to guide our research on 
educational change we needed: 

1. To develop a model of the innovative process that centers on implementa- 
tion. 

2. To define outcome measures that assess the effectiveness of implementa- 
tion and the extent to which the change agent projects meet the policy 
goals of project continuation and dissemination. 

3. To identify factors affecting the innovative process and, consequently v the 
outcomes of innovative projects. * 

% ■ 

1 See Vol. I for a discussion of the literature on this point. 
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STAGES OF INNOVATION 

We view the innovative process as consisting of three stages: initiation, im- 
plementation, and incorporation. The initiation stage in the life of an innovative 
project occurs when local school officials conceive and formulate plans, seek re- 
sources, and make decisions about which projects they should select and support. We 
hypothesized that the support and commitments made in the initiation period affect 
what happens when project implementation begins. 

A crucial stage is the implementation stage, when the project confronts the 
reality of its institutional setting and project plans must be translated into practice. 
Many innovative projects fail or produce disappointing results because they are not 
implemented according to plan. But the issue of implementation is often more subtle 
and complicated than mere fidelity to some specific blueprint for reaching a set of 
educational goals. We hypothesized that local school systems are so structured that 
in order to implement significant innovations there must be a process of mutual 
adaptation. 2 The initial design of an innovative project must be adapted to the 
particular organizational setting of the school, classroom, or other institutional 
hosts, and, at the same time, the organization and its members must adapt to the 
demands of the project. Many educational innovations may fail to have desirable 
effects because the project is not tHapted to the institutional setting or vice versa 
during the implementation stage. 

The term incorporation is used to denote the final stage in innovation— when 
an innovative practice loses its "special project" status and becomes part of the 
routini^ed behavior of the LEA. Incorporation involves the most serious commit- 
ment on the part of the district, as federal "seed money" is withdrawn and decisions 
must tie made about not only whetherbut also what components of and on what scale 
a project should be continued within the district. We hypothesized that this decision 
may' involve more than the success or failure of the project during its trial period. 
Economic, political, and organizational pressures and constraints may play major 
roj'es in determining the innovation's future. 

' These three stages— initiation, implementation, incorporation— involve some- 
what different activities and decisions, and the significance of actors and issues also 
/changes from one stage to another. We believe that a key to designing and assessing 
federal, state, and local policy lies in understanding how the stages of innovation- 
work in different locations, for different innovations, and for the various change 
agent programs. 

DEFINING "OUTCOMES" 

As the innovative process is better understood, it becomes possible to analyze 
innovations by measuring the "outcomes" of a project and ideij|^ng factors in- 
fluencing these outcomes. But defining which project outcomes to measure had to 
be done carefully. 



2 Volume I defines an innovation as a plan with a statement of goals and means designed to change 
standard behavior, practices, or procedures Many educational innovations tend to have abstract goals, 
to be vague abou means, and to be uncertain about the relationship between means and ends. Such 
uncertainty makes it desirable for the innovation to become developed, revised, or, in short, adapted to 
the realities of its institutional setting. Accordingly, we define implementation as the change process that 
occurs when an innovative project impinges upon an organization. 



Although student outcomes might be the ultimate indicator of the effectiveness 
of ai innovation, it was both premature and inappropriate to use them as measures 
in this study. The projects we studied were generally new to the district or school 
and usually needed time to be developed before an accurate assessment of their 
long-run effects on students could be made. Moreover, the change agent projects had 
such diverse goals and ambitions that comparisons across projects on absolute or 
content-free student performance measures were meaningless. For example, an 
increase of a half grade level in cognitive test scores for a remedial mathematics 
project in an elementary school cannot be reasonably compared with a measure of 
greater awareness of career opportunities for a high school career education project. 

- Importftn^-conceptual reasons-al so lead u s-ttrTneastmrprojec t "outcomes" o ther - 
than student outcomes. Because projects must go through the complex and uncer- 
tain process of implementation before they can affect students, it makes sense to put 
first things first and to measure the effectiveness of implementation before examin- 
ing potential student impacts. 

Moreover, projects may "mutate" during implementation. Unless one can pre- 
dict how the project changes, it is not possible to determine whether variations in 
student outcomes are the result of the initial project plans or of the interplay of the* 
project with its institutional setting. In short, for the purposes of this study, we 
focused on implementation and developed the following measures of the effective- 
ness of a project's implementation: 

1. Perceived success; the relative extent to which project participants be- 
lieved that goals were achieved. 

2. Change in behavior: the type and extent of change in teacher and admin- 
istrator behavior as perceived by participants. 

3. Fidelity of implementation: the extent to which the project was imple- 
mented as originally planned. 3 

These measures are designed to indicate how project, school, and district staff 
assessed the project's implementation. Moreover, in combination, they enable us to 
make overall judgments about factors affecting implementation. For example, 
project characteristics that are positively related to fidelity of implementation may 
be negatively related to teacher change. This result would be consistent with the 
following hypothesis: Unless the project deviates from its original proposal, that is, 
unless it adapts, teacher change may not occur. As another illustration, project 
characteristics that are positively associated with perceived success but negatively 
associated with teacher change might reflect (depending on the characteristics in 
question) "successful" but trivial projects. 

Another policy relevant "outcome" is continuation-— the extent to which the 
LEA continued project activities after federal funds were withdrawn. This measure 
provides a critical test of the value of change agent projects because federal planners 
anticipate that change agent funds provide seed money. Moreover, continuation 
may represent in effect a local market test of the merit of the innovation. It) this 
first phase of the Rand change agent study, we examined projects in their last years 
of federal funding and, hence, were able to measure only expected continuation. 4 

1 The difficulty of implementation was another dependent variable used in conjunction with the above 
indicators of the project s implementation. 

4 Another outcome is dissemination— the extent to which the project is spread from its original site 
Dissemination can involve the spreading of all or some project practices both within the district and 
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IDENTIFYING FACTORS AFFECTING IMPLEMENTATION 
AND CONTINUATION 

We hypothesized at the outset of the study that the innovative process and, 
consequently, project outcomes are largely determined by the interplay among 
characteristics of the project, the institutional setting, and federal policies. A major 
objective of our research was to weigh the influence of these factors on implementa- 
tion and continuation. 

To do so, we identified components of the general factors and analyzed their 
significance for project outcomes, controlling for other components. For example, we 
-distinguished the educational treatment of a project from-the-projecL'aimpLejnenta^ 
tion strategy, which we defined as the choices made to implement the project. The 
findings, which are reviewed in Sec. IV, consist of estimating the extent to which 
educational treatment as compared with implementation strategy determined im- 
plementation outcomes. 

outside of the district. This project outcome is a central focus of the second phase of our research. The 
first phase of the study did not systematically collect data about dissemination, except for observations 
in some fieldwork sites. See Vol" III, Sec. IV, for a discussion of these preliminary observations. 
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III. THE PROCESS OF INNOVATION 



A change agent project proceeds through three distinct stages— initiation, im- 
plementation, and incorporation— as it deyelops from an innovative plan to an 
operational reality. Our fieldwork provided evidence that in each of these stages 
there were certain typical processes and decisions by principal actors and, moreover, 
that the institutional setting heavily influenced the way these processes worked. 



INITIATION 

We observed four factors that interacted to spur the initiation of change agent 
projects: the presence of a "good" idea, the availability of federal funds, local needs, 
and the incentives of individual actors. 

This interaction in particular settings defined initiation processes that we found 
could be characterized into two ideal types: opportunism and problem solving. 1 The 
contrasting motivations that characterized these different initiation processes con- 
tinued to play a pervasive role in the implementation and thus in the outcomes of 
the innovations. 

Projects generated essentially by opportunism seemed to be a response to avail- 
able funds and were characterized by a lack of interest and commitment on the part 
of local participants— from district administrators to classroom teachers. As a re- 
sult, participants were often indifferent to project activities aud outcomes, and little 
in the way of serious change was ever attempted— or occurred. 

The problem-solving motive for projects emerged primarily in response to local- 
ly identified needs and was associated with a strong commitment to address these 
needs. Federal funds were viewed as a way to support the loca! solution— one which 
often broke new ground in local educational practice. 

It is important to note that the "search for alternatives" traditionally assumed 
to be characteristic of the problem-solving approach to innovation did not occur. The 
designers of change agent projects that were typified by problem-solving motivation 
did not search. Instead, they used information or treatments that were already 
known to local district personnel. f " 

It is possible that local administrators did not search outside of their districts 
for "better" treatments or technologies because they intuitively feel that the success 
and suitability of an innovation depend primarily on local conditions, a view that 
our evidence supports. Thus, local administrators are likely to be skeptical about the 
reported "success" of educational methods in other districts and tend to rely on the 
advice of local professionals who have a thorough knowledge of particular local 
conditions. * 

The opportunity-based projects also generally used available information or 
current knowledge, except when they accepted— essentially in toto— the design and 
advice of outside experts. Projects imported into the districts from the outside usu- 
ally failed to gather the support of LEA staff. Not only were these projects unable 

1 See Vol. Ill, Sec. II. 22 
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to elicit initial commitment but, because they were not locally conceived, they could 
not generate,staff involvement during implementation. Generally, whether a project 
was initiated primarily to solve a local problem or largely in response to an oppor- 
tunity, the involvement of all key participants in its early or formative stages was 
important ,to implementation. 
* 

/ 

IMPLEMENTATION 

Contrary to the assumption underlying many change strategies, implementa- 
tion didotiot involve merely the direct application of a technology. Implementation 
was an organizational process that implied interactions between the project and its 
setting; thus, it was neither automatic nor certain. 

We observed three types of interactions — defined by the extent to which the 
project was adapted to the institution or vice versa— that characterized implementa- 
tion processes: 

1. Adaptation of both the project design and the institutional setting; we call 
this mutual adaptation. 

2. No adaptation on the part of either project or setting; this lack of change, 
which we call nonimplementation, typically occurred in instances where 
the district played an indifferent host or projects were overcome by im- 
plementation problems. 

3. Project adaptation to the indifference and resistance to change on the part 
of project participants but no change by participants themselves; this one- 
way process could be called cooptation of the project by its host. 

A fourth type of implementation could be imagined; that is, some behavior, activi- 
ties, and practices of the staff could change as a result of the project even though 
the project itself would not be modified; this could be called technological learning. 
This situation could describe implementation in the instances of projects involving 
"pure" technologies or comprehensive packages such as the so-called teacher-proof 
curricula. However, although logically possible, pure technological learning ap- 
peared not to occur in practice. Instead, these highly prescriptive techniques were 
either adapted to local needs and interests or were not implemented at all. 

The type of implementation process— mutual adaptation, cooptation, or nonim- 
plementation — that occurred for any particular project depended on three consider- 
ations: the motivations and circumstances involved in its initiation, its substance 
and scope of proposed change, and its implementation strategy. 

Opportunistic projects tended, other things being equal, to either be coopted 
during implementation, or to undergo a symbolic type of nonimplementation. For 
example, we observed career education projects that involved the addition of peri- 
pheral enrichment materials to the standard curriculum. In these cases, implemen- 
tation was essentially pro forma; it constituted in effect nonimplementation of the 
ideas underlying career education. That the participants usually reported success 
in meeting their project's goals— without change* in teacher behavior — was often 
testimony to the lack of ambition of their innovation. They seemed to be indifferent 
to the projects. oo 
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Some projects that were initiated in a problem-solving manner also experienced 
cooptatipiror nonimplehentation. but they tended to have different characteristics. 
For example, some Right-To-Read projects experienced nonimplementation because 
they simply broke down as project participants attempted to apply the prescribed 
management strategies and could not cope with unanticipated requirements. An 
illustration of cooptation associated with a problem-solving approach occurred for 
a top-down staff development project involving differentiated staffing where, despite 
commitment from administrators, staff members adapted the project to their own 
needs and did not significantly modify their own behavior in accord with the pro- 
ject's intent. 

In all the cases we observed, mutual adaptation occurred only if the project was 
preceded by attitudes and commitments associated with problem solving. Thus, a 
problem-solving initiation may be a necessary condition for mutual adaptation. 

Mutual adaptation could involve a variety of adjustments— for example, reduc- 
tion or modification of idealistic project goals, amendment or simplification of 
project treatment, downward revision of ambitious expectations for behavioral 
change in the staff or of overly optimistic effects of the project on students, and 
unanticipated changes in standard practices by or relationships between stafTand 
administrators. These adjustments often caused difficulties and did not invariably 
lead to a full achievement of the project's goals. But they typically increased the 
likelihood of changes in teacher and organizational practices. 

In addition to greater prospects for mutual adaptation arising frofn a problem- 
solving approach to change, the extent of mutual adaptation that occurred depended 
on the substance and scope of change proposed by the project design, particularly 
on how complex and specific the methods and goals were, and flexibility in coping 
with unanticipated implementation problems. The most extensive mutual adapta- 
tion took place in projects, such as open-classroom innovations, that were highly 
complex, relatively unspecified in terms of prescribed treatment, and required a 
significant amount of change on the part of teachers and administrators. 



INCORPORATION 

Continuation of federally sponsored projects after the end of federal financial 
support is usually thought about In indivisible and institutional terms; that is, a 
project is considered to be "continued" if a decision is made by the district to carry 
on the original "package" or "program" of goals and treatment. But our research 
suggested that continuation is not always such a straightforward question; nor is it 
always the result of formal district decisions. 

Rather than simply a yes-no decision: to continue a project on district funds, we 
found that a process of incorporation takes place in which portions of projects may 
become part of the ongoing activities of the schools and thus may be maintained. 
Moreover, incorporation had two aspects. At the classroom level, teachers or princi- 
pals often planned to assimilate parts of a change agent project into their regular 
routine, with or without formal project affiliation or district sanctions. At the school 
district level, incorporation implies LEA support— financial, organizational, and 
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political — to continue the project when it loses its special project status. We found 
little evidence, at either level, of direct federal policy influence on incorporation. 2 

For the sake of clarity, we refer to continuation as an "outcome" measure of the 
extent to which a project persists at any level in the local school district after the 
end of federal funding; incorporation refers to the process itself. 

At the classroom level, we found that projects that replaced existing practices 
seemed more likely to continue the new practices than those that merely supple- 
mented or added on to the existing curriculum. Continuation was more likely under 
the following conditions: an emphasis on training rather than on the introduction 
of new technology, training focused on practical classroom issues rather than on 
theoretical concepts, and local development of materials rather than reliance on 
outside consultants. Teachers participating in projects that successfully implement- 
ed these training or developmental activities reported that they "could never go 
back" to traditional classroom roles and behavior. Conversely, change agent pro- 
jects, such as the career education projects, which typically simply added new activi- 
ties or materia%to a district's repertoire, were rarely expected to be continued 
(unless the SEA mandated some type of similar categorical program in the district, 
as was sometimes the case with bilingual and career education). Our observations 
suggest that the ancillary materials employed by these projects were likely to fall 
into disuse without the active encouragement of a special project staff and explicit 
use by another project. In the case of add-on projects, it st ^ms likely that when 
special project status and staff go away with the last federal check, these additional 
materials and supplementary activities will be discontinued. 3 

The only effect federal change agent programs had on expected project con- 
tinuation was an indirect one that derived from the training and developmental 
activities of projects. Continuation was most likely to take place in Title III projects 
because classroom organization and staff development projects were more likely to 
be funded by Title III. However, vocational education projects did not meet these 
incorporation "conditions" and were generally not expected to survive the disap- 
pearance of federal funds. 

At the district level, a pattern emerged concerning the relationship between 
expected continuation and initial motivation and support. Decisions or expectations 
about individual or district continuation of project activities appeared to closely 
parallel the decisions or motivations that underlay project initiation. Projects that 
were initiated with strong district support, and that were also seen as a solution to 
a particular problem, were generally expected to be continued. However, projects 
that represented an opportunistic response to available dollars and received little 
or no support from district administrators were usually expected to wither away, 
even when project objectives were met. 

More specifically, the data provide evidence that superintendents weigh four 
general concerns in reaching decisions about continuation — the project's "success" 
during implementation, the centrality or importance of the educational needs 
served by the project, the resources, required by the project, and the organizational- 
political forces inhibiting or promoting the innovation. In particular, if a project was 

2 These findings must be treated as preliminary. We studied projects in their last years of federal 
funding and thus > could only examine expected continuation. The next phase of Rand s research wUl focus 
on projects after the withdrawal of federal support. 
' 3 See Vol. Ill, Sec. IV. 25 
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perceived as central and successful, had the support of the staff, and was not too 
expensive, it was likely to be continu *d. Such projects were often initiated as the 
result of the need to solve a problem within the district and often replaced tradition- 
al teaching activities with new departures (e.g., classroom organizational changes). 
Projects initiated in an opportunistic fashion typically became add-ons and disap- 
peared with the termination of federal funds. 4 Evaluation evidence did not appear 
to play a major role in continuation decisions. We observed "successful" projects 
that were expected to be terminated, and "unsuccessful" projects that were to be 
continued. In short, the initial patterns of motivation that underlay initiation per- 
sisted; support or commitment was not altered by evaluation data. 



Thus, each stage in the process of change— initiation, implementation, and 
incorporation— was described in terms of the interplay Jbetween the characteristics 
of the project and its local setting, The specific relationships and patterns of motiva- 
tions discussed above serve as both findings and hypotheses that allow us to examine 
and assess the significance of factors affecting the outcome^of innovative projects. 

4 See Vol. IL Sec. V, 
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IV: . FACTORS AFFECTING IMPLEMENTATION 
/ AND CONTINUATION 



Wp hypothesized in our theoretical approach that three general factors can 
influence the course of innovative projects— project characteristics, the institutional 
setting, and federal policies. The preceding analysis of the innovation process sug- 
gests key components of these general factors", which are listed; along with the 
previously discussed measures of the effectiveness of implementation and continua- 
tion, in Table 1. - 

Project characteristics listed in Table 1 consist basically of four elements: edu- 
cational treatment, resource level, scope of proposed change, and implementation 
strategy. Although most studies distinguish between the project's educational treat- 
ment and its resource level (e.g., level, of federal funding, number of students served 
by the project, and per-pupil expenditure), many studies attempting to relate project 
characteristics to outcomes fail to distinguish the treatment from the scope of 
change contemplated by the would-be innovators. Yet such dimensions as the com- 
plexity and the amount of change required by a project can be expected to place 
different demands on the institutional setting and thus may have strong effects on 
project outcomes. Moreover, project evaluations seldom differentiate the education- - 
al treatment or technology from the (usually implicit) implementation strategy 
selected to carry out the treatment. Yett>ur research provided us with many illustra- 
tions of the same basic treatment being implemented in contrasting ways in differ- 
ent school and district settings, resulting in different outcomes. • . 

Another general factor assumed to affect innovation is the institutional setting. 
Experience and common knowledge suggest that schools and school districts differ 
from one another in many ways, dome statistical studies have analyzed thp back- 



table 1 

Factors That Can Affect Innovative Projects 



S 



Factors or Independent Variables 

Project Characteristics 

Educational treatment or technology 
Resource level 
Scope of proposed change 
Implementation strategy 

Institutional Setti 4 

Organizational climate and motivations 

of administration and staff 
. Characteristics of school, district, and 

principal actors 

Federal Policies 

Federal change agent programs* objectives 



/ 



Dependent Variables or Project "Outcomes* 



/ 



and management strategies 



Fidelity of implementation 
Porceived success 
Change in behavior 
(Expected) continuation 
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ground and demographic characteristics of a projects school, school district, and 
paiticipant> Hut out Muds of the proce** of innovation suggests that the organiza- 
tional climate and the moti\ation> of principal actors also can play critical roles in 
project outcomes 

The final factor that might affect innovative project* i* federal policies The 
Hand change agent stud\ examined lour federal progiams designed to promote 
educational change, each with a different nv>»:*gemem MrategN »"d a different set 
nf pnonties This diversit\ prouded an opportunit\ to compare the innovations 
funded In difieient program^and toa>se>s the extent to which differences in federal 
program strategies and priorities could account for variations in the innovative 
process and project outcomes However, our priman concern was not with evaluat- 
ing am particular program but with using program comparisons as one measure of 
judging the effectiveness of polia instruments common to these change agent pro- 
grams 

This section reviews, in turn, our findings about the nature and significance of 
thfse independent variables for effective implementation and continuation. 



EFFECTS OF PROJECT CHARACTERISTICS ON PROJECT 
"OUTCOMES" 

Educational Treatment or Technology 

* 

Educational treatment traditionally has been thought to be a major factor ex- 
plaining the outcomes of innovative projects. Our sample of projects evidenced a 
great diversity of treatments ranging from open educational efforts to computer- 
assisted instruction. To reduce this variety, we collected survey'data on the specific 
techniques used by a project and grouped these techniques statistically. Using this 
procedure, we developed five factors that represent the basic educational approaches 
in our sample— classroom organizational changes, enrichment techniques, intensive 
traditional staffing, school administrative changes, and behavioral modification 
techniques. 1 Analysis of these educational approaches showed that, other things 
being equal, they explained relatively little of the variation in implementation 
outcomes and in continuation. 2 

This finding is subject to measurement error. Yet our fieldwork supported the 
statistical evidence. For example, we observed the same mathematics program in- 
stalled in a district in four schools having comparable student and staff character- 
istics. But the project was implemented in dramatically different ways at each site. 
We believe these results reflect— and tend to confirm— the mutation hypothesis that 
the treatment is generally adapted and altered by the schools during implementa- 
tion, f 

This evidence of the lack of a consistent correlation between treatment and 
outcome suggests a simple but highly important implication, particularly for project 

1 ThisV^cedure used factor analysis. For statistical details, see Vol, II. Sec II. 

2 See Vol. II, Sees. IV and V. Several exceptions to this overall statistical pattern occurred A signifi- 
cant exception involved classroom organizational changes: Title III projects with high levels of classroom 
organizational changes were more likely to be continued than other Title III projects, even though they 
were more difficult to implement and were perceived as no more successful, at least in the short run. 

28 
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evaluators: The predictors of effective implementation are likely to lie in those 
project characteristics promoting or inhibiting adaptation to the institutional set- 
ting—the implementation strategy— rather than in the educational treatment or 
technology itself. 

Resource Level 

Many projects in our sample received financial assistance from state, local, and 
foundation sources in addition to federal funds. Both the absolute amount of money 
available to projects and the project per-pupil expenditure varied considerably. 
Nonetheless, other things being equal, variations in the funding level, the number 
of students served, and the concentration of funding had small and generally not 
significant effects on project outcomes. 3 

This finding casts doubt on the possibility of using outside funding, whether 
administered by the federal government or state education agencies, as a finely 
tuned policy instrument. 4 However, it does not imply that the injection of federal 
funds was inconsequential. On the contrary, our respondents strongly indicated that 
many of the innovations attempted would not have been possible without initial 
outside financial support. Our sense of the general political and economic con- 
straints on school districts supports this view. 

Substance and Scope of Proposed Change 

The evidence so far suggests that neither variations in educational treatments 
nor resource levels greatly affected implementation outcomes. A characteristic of 
innovations that did have major effects was the scope of change proposed by the 
project design. 

Innovative projects in the Rand sample differed considerably not only in terms 
of their substance (e.g., heading versus career education) but also in terms of the 
scope of the proposed chMnge. Some projects consisted of highly specific and uncom- 
plicated additions to the curriculum or technological supplements to standard prac- 
tices. As such, they neither required much change from teachers or administrators 
nor conflicted with usual procedures and practices. Other projects were exactly the 
opposite. To describe this aspect of a project, we 'define four dimensions— the pro- 
ject's centrality (i.e., how close the goals of a project were to major educational 
objectives of the district), the nature and amount of change required, the project's 
complexity, and its consonance (i.e., the fit between the project's goals, values, and 
practices and those of the schools and district). 5 

These important dimensions of the scope of change were, of course, difficult to 
measure and, hence, our findings are tentative. Yet they had sych pervasive influ- 

3 See Vol II/Sec. IV. Three significant exceptions may have important implications for federal 
planners, even though the statistical effects are small. Projects with large target groups were negatively 
related to teacher change, particularly in elementary schools. However, the more concentrated the 
funding was on Title III projects, the more likely was teacher change. And the more expensive a project 
was, the less likely it was to be continued. 

4 This finding must be qualified. Although the per-pupil expenditure reached high levels (oyer $5000 
per student per year) for a small percentage of the projects in the sample, the federal funds provided were 
^relatively small percentage of most school district budgets. Therefore, very large increments of outside 
funds (e.g., 30 percent of a district's budget) might have major effects on project outcomes. _ . 

1 fW> Vol. L Sec. Ill; Vol. II, Sec. V; and Vol. Ill, Sees. I and III. 
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ence on the process of innovation and its outcomes that we feel justified in emphasiz- 
ing their importance. 

Whether an innovation was perceived as central to the district's priorities or as 
ancillary appears to have affected the interest and commitment of project partici- 
pants at all levels. Projects initiated in a problem-solving fashion were, by their very 
nature, central, whereas opportunity-based projects tended to be ancillary. We 
found that the more central an innovation was, the more likely it was to be con- 
tinued by the district using its own resources. This finding is particularly significant 
because it held even controlling for the cost and perceived success of the project. 
That is, projects with high district priority were likely to be continued even in cases 
when they had not been relatively successful during tHeir temporary federal fund- 
ing period and when they were expensive for the district. 6 Ancillary projects tended 
to-be add-ons to district practices and were not likely to be continued. 

Significant change in teacher behavior was not likely to occur when it was not t 
required by the innovation. Projects requiring teacher change and extra effort were 
difficult to implement and generally did not adhere to the initial project design. 7 
These serious change attempts were often perceived as relatively less successful in 
achieving their stated project goals during the period of temporary federal funding 
than other less ambitious, more narrowly focused projects. The reason for this 
disparity between teacher change and short-run success may be due to the necessity 
for new behavior to be incorporated before its effects can be realized. . 

Complexity is closely associated with other dimensions of change and is thus 
difficult to measure. Nonetheless, we observed three aspects of project complexity 
that had different effects on 'outcomes. First, some projects were structurally com- 
plex. These projects attempted a comprehensive innovation that spanned many 
grade level* or tried to include all classrooms in particular grade levels in a district. 
The most complex— those covering both elementary and secondary schools— were 
very difficult to implement, were not perceived by project participants as being 
successful, find were unlikely to be continued. 8 This suggests that structural com- 
plexity requiring a great deal of coordination across school grades and levels is not 
likely to eventuate in successful projects. Such projects often broke down because 
they attempted too much too soon. 

In contrast, a second aspect of complexity involves the treatment. A strong 
' impression gleaned from our field pbservations was that narrow treatments did not 
lead' to broad-tased or enduring change in teacher behavior. This suggests that 
innovations involving a comprehensive area of curriculum or requiring an overall 
change in teacher behavior are more likely to induce Change, other things being 

equal. * , 

A final type of complexity involves the integration of project activities into the 
ongoing procedures of the school or school district. For examplfe, an innovation such 
as a differentiated staffing project requires schoolwide scheduling chahges.and dislo- 
cations of faculties. These projects were difficult to implement unless they had the 
active support of the school. Similarly, integration was also important at the class* 
room level. For example, projects that included a heavy teacher training or staff 
development component, but that tfid not tie these new skills and behavior to ongo- 

• See Vol. II, Sec. V. 3Q 
7 See Vol. II, Sec. IV. 

* See Vol. II, Sees. IV and V. 
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ing classroom responsibilities seldom succeeded in promoting significant or endur- 
ing change in teacher behavior. Conversely, projects— such as the classroom organi- 
zation projects— that integrated change in training with expectations for concomi- 
tant classroom behavior were likely to result in basic change in teacher activities. 

The final aspect of the scope of change attempted by a project involves the 
notion of consonance. If the values and goals implicit in a project's design were not 
congruent with those of project participants, the innovation was likely to be either 
symbolically implemented or not implemented at all. For example, career education 
projects seemed to be particularly susceptible to this lack of consonance. In contrast, 
those innovations characterized by complex treatments involving integration of 
different elements of the organization appeared to require the kind of support from 
district officials and commitment from the staff that only comes when their values 
are consonant with the project's philosophy and goals. This appears to be particular- 
ly important when the proposed innovation represents a major departure from 
standard district practice. One of our fieldwork staff development projects illustrates 
this dramatically. Midway into the project's implementation, a change in school 
board membership resulted in a district-level reorganization, and the new adminis- 
trators had values in conflict with the precepts of the staff development project. 
Consequently, the project lost district support and floundered. 

Implementation Strategy 

A project's implementation strategy results from many choices about how to 
implement its goals and educational treatment. Decisions about the type and 
amount of planning, the location of the project, and about who'should participate 
(and to what extent they should participate) are examples of such choices, and define 
ii\effect how an educational treatment is put into practice. Implementation strate- 
gies are distinguishable from project treatment. The educational method chosen for 
a project (i.e., diagnostic/prescriptive reading techniques or classroom organization) 
is different from the strategy for implementing that methocl. For example, no two 
reading projects employ quite the same process or strategy for achieving their 
almost identical objectives. 

Data from the survey and the fieldwork clearly indicate that (after controlling 
for variations in institutional settings, in federal programs, in educational methods, 
and in project resources) the implementation strategies selected to carry out a , 
project vitally influence the innovative process and project outcomes. 9 

In 'particular, the strategies that significantly promoted teacher change in- 
cluded staff training, frequent and regular meetings, and local material develop- 
ment. The absence of any one of the above elements was likely to reduce the per- 
ceived success and the amount of teacher change on projects. The lack of teacher 
participation in day-to-day implementation decisions also reduced perceived success. 

Our fieldWork suggests that each project employed its 'own combination of 
strategic choices that defined in effect its particular implementation strategy. Thus, 
it is more meaningful to discuSs how— and why— the various individual strategic 
choices interact with each other to form a "successful" implementation strategy. We 
" believe that a successful implementation strategy is one that promotes mutual 
adaptation. By doing so^t is the most likely to produce significant and persistent 

• See Vol. II, Sec. IV, and Vol. Ill, Sec. III. 31 



teacher change and holds the best prospects Tor the long-run achievement of project 

goals. 

We identified the following elements to be the main components of an im- 
plementation strategy that promotes mutual adaptation. 

Adaptive Planning. Projects varied considerably in the amount of planning, 
they did, and it would appear that in their first year the more successful project 
avoided the extremes of almost no planning and of almost all planning. But other- 
wise, the amount of planning was not significantly related to project outcomes. 
Indeed, it appears that the amount of planning seemed less important than whether 
the quality of planning matched the needs of the project and its participants. 

Although the resources spent on planning had little effect on project outcomes, 
the nature of the planning process had a major effect. Planning activities that were 
flexible, adaptive, and congruent with the i^ature of the project were more likely to 
result in well-implemented innovations. By "flexible" and "adaptive" planning, we 
mean planning that established channels of communication, set forth initial goals 
and objectives with the assistance of a representative group of prospective project 
participants, and maintained a continuing process of planning. Frequent and regu- 
lar staff meetings contributed significantly to project success because they made 
planning a continuous process. These meetings provided a forum for reassessing 
project goals and activities, monitoring project achievements and problems, and 
modifying practices in light of institutional and project demands. Planning, in this 
instance, had a firm base in project reality f so that issues Could be identified and 
solutions determined before problems became crises. Meetings also strengthened 
staff morale^ established a sense of project cohesiveness, and broke down the tradi- 
tional isolation of the classroom teacher. 

Staff Training Keyed to the Local Setting, Projects also differed greatly in 
the amount, timing, and type of training for project staff Training was significantly 
related to project outcomes only when it was tied to the specifics of project operation 
and to the practical day-to-day problems of the project participants. For example, 
we saw that the effectiveness of training was conditioned by the training format and 
by who did the training. Teachers strongly preferred very concrete "how-to-do-it" 
workshops given by local personnel (as opposed to a more general lecture, inspira- 
tional format). The projects that were implemented most smoothly had either a 
project director or district resource personnel whose understanding and experience 
.(both in project methods and in the local setting) enabled them to make specific 
suggestions to help teachers, implement the project. Teachers said that outside 
technical assistants performing a similar consulting role were ineffective and disap- 
pointing. 40 

Local Material Development Material development activities ranged from 
careful assessment and "repackaging" of existing products to producing from 
scratch a wide range of project materials. These development activities can play an 
important role in successful project implementation and, subsequently, in project 
outcomes. The value of producing one's own project materials may not lie principally 
in the merits of the final product, but in the activity of development itself. The 
/ 

10 One strategy that was not significantly related-to outcomes was extra pay given as an incentive 
for teacher training. Our fieldwork observations suggest that money and other tangible rewards were not 
effective in inducing teachers to acquire new skills if their own professional interests or concerns did not 
lead them to see such new learning as important. ^ 
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exercise of "reinventing thf wheel" can provide an important opportunity for staff 
to work through and understand project precepts and to develop a sense of "owner- 
ship" in project methods and goals. Without this "learning by doing," it is doubtful 
that projects attempting to achieve significant teacher change would be effectively 
implemented. 11 

Critical Mass. Although project participants did not show much resistance to 
innovation, particularly when there was a strong commitment on Hie part of the 
district, nonproject personnel sometimes impeded prefect implementation. When 
project teachers felt "isolated" (and unappreciated), negative or indifferent attitudes 
from nonparticipants eroded staff morale and constituted a pressure for the project 
teacher to "give up;" \ 

Apparently, a critical mass of project participants is necessary to build the 
support and morale of the project staff. Furthermore, critical mass of project staff 
in a given site is able to establish a norm for change in the setting, rather than 
making, project teachers seem to be deviant. 

In sum, these components— adaptive planning, staff training keyed to the local 
setting, local material development, and the establishment of a critical mass— were 
the key elements of an implementation strategy that promoted mutual adaptation. 
For this reason, innovations employing these elements often had a slower start-up 
and a difficult implementation but were more likely to result in significant and 
enduring teacher change. 



EFFECTS OF INSTITUTIONAL SETTING ON PROJECT "OUTCOMES" 

The preceding discussion of the process of change and of implementation strate- 
gies foreshadows a major conclusion of this study: An innovation's local institutional 
setting has the major influence on its prospects for effective implementation. Al- 
though we could not collect data on all aspects of the institutional setting, our 
statistical analysis as well as our fieldwork clearly showed that project outcomes 
depended more on the characteristics of the project's setting than on any other 
factor. 

In particular, the local organizational climate and the motivations of project 
participants had major effects on perceived success and on change in teacher behav- 
ior. More specifically, high morale of teachers at a school, the active support of 
principals who appear to be the "gate-keepers" of change, the general support of the 
superintendent and district officials, and the teachers' willingness to expend extra 
effort on the project all increased the chances of teacher change and perceived 
success. The attitudes of administrators in effect tell the staff how seriously they 
should take"project objectives. Unless the project seems to represent a district and 
school priority, teachers may not put in the extra effort and emotional investment 
necessary for successful implementation. Thus, when these elements were not in 
evidence, projects were likjely to break down or be implemented symbolically with- 
out significant change. 
' Organizational climate and individual commitment are important because sig- 
# 

11 We dc not have direct evidence as to whether the quality of locally developed materials "improved" 
the curriculum. However, project participants consistently reported that locally/ developed materials 
were better for their needs than those that they replaced, qo 
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nificant innovations often require more than the mere installation of a promising 
educational method, technique, or technology. They usually assume that individuals 
alter their traditional patterns of behavior. In other words, the institution must 
adapt to the demands of the change agent project even as the project adapts to its 
environment. 

Because' the school organization must adapt if significant change is to take 
place, the receptiveness of the institutional setting to the change agent project 
seemed to be a necessary condition for successful implementation. Naturally, im- 
plementation was difficult in a hostile environment, but indifferent settings also 
failed to provide necessary support. 

Indifferent and unreceptive environments were frequent in our sample of pro- 
jects attempted in secondary schools. Innovations in elementary schools were more 
likely to be successfully implemented and to result in teacher change than projects 
in high schools, junior high schools, or those that cut across elementary and high 
school levels. Change agent projects that included the higher grade levels ex- 
perienced severe management and administrative problems as well as teacher resis- 
tance. For example, Right-To-Read projects consistently encountered resistance at 
the high school level as they attempted to persuade science or history teachers to 
view themselves as teachers of reading. The same thing happened with career 
education projects. Project managers could generate little interest in "new ideas" 
among secondary school teachers of "solid subjects" who perceive themselves as 
having large intellectual and emotional investments in academic purity. In short, 
this tendency toward strict professionalism among secondary school teachers (along 
with the compartmentalization of the curriculum and classroom scheduling) may 
not have provided the organizational conditions necessary for significant change 
efforts. 

In contrast, a receptive institutional setting provides explicit, steady support for 
change agent efforts. As such, a receptive institutional setting is a necessary but no£ 
sufficient condition for effective implementation. Mutual adaptation— which we be- 
lieve is the key to serious change — requires an effective implementation strategy, 
one that takes advantage of institutional support. Indeed, the components of the 
implementation strategy that we found to be most effective — adaptive planning, 
stafftraining keyed to the local setting, and local material development— were those 
that enabled the support and commitment of administrators and staff to be fully 
engaged. 



EFFECTS OF FEDERAL POLICIES ON PROJECT "OUTCOMES" 

Because federal policies provide external inputs to the course of an innovation, 
their effects should be analyzed differently from such internal factors as implemen- 
tation strategies and organizational climate that are an intrinsic part of the inter- 
play characterizing the innovative process. Federal policies might affect each of the 
stages of innovation direcfely, and, moreover, a federal policy effect on the ^initiation 
stage might be played out indirectly during implementation and incorporation. 
These direct and indirect influences were analyzed by comparing effects on each 
stage for each federal program. 

We found that federal policies primarily affected only the initiation stage but 



22 



that these initial influences were not reflected in implementation or incorporation; 
consequently, federal policies had little influence on project outcomes. 

In particular, reports from local project staff as well as our field observations 
clearly indicate that the availability of federal funds made many projects possible 
that simply could not have been initiated solely on a district's limited budget. 
Moreover, the three categorical programs usually promoted local projects that were 
congruent with their federal categorical priorities. 

Furthermore, statistical analyses indicated that the guidelines, the program 
characteristics, funding, and the priorities of federal programs did affect initial 
project design, but not much. Whatever the motives of school districts— seeking 
opportunities to gain federal money or attempting to solve their problems by initiat- 
ing innovative projects— it seems clear that some educational methods were more 
likely to be employed than others because some federal programs fostered these 
methods, either in their administrative guidelines or in'their focus. Thus, comparing 
Right-To-Read projects with Title III reading projects, we find that behavioral 
modification techniques and concentrated traditional staffing were \ess likely to be 
adopted by the locally initiated Title III reading projects. Increased levels of enrich- 
ment were associated with vocational education and Title VII when compared with 
Title III projects of a similar form/However, federal funding opportunities appeared 
to have less effect on the extent to which local districts elected to employ such locally 
coriceivecJ%rojects as innovation in classroom organization. 

Whereas the federal program effects on initiation were direct, their influence 
on implementation was necessarily indirect. The change agent policies did riot 
provide explicit assistance or intervention during the hnpjefnentati'on phase. Yet 
the different guidelines and management strategies could affect the way projects 
were implemented. The data suggest, however, that the differences between the 
programs.^xplained only a small amount of the variation in implementation out- 
comes and continuation decisions. 

Using four measures of project outcomes— fidelity of implementation, perceived 
success (percentage of goals achieved), teacher change, and expected continuation— 
we directly compared the mean outcomes of projects funded by the various federal 
change agent programs. 12 The federal programs differed little from one another on 
these measures. 

This finding suggests that federal change agent programs had approximately 
equal effects on project oytcomes, despite their different management strategies. A 
simple and yet appropriate explanation for equivalent program effects is that the 
policy common to these programs had limited influence on implementation and 
incorporation. 13 

These findings that federal policy had little influence on project outcomes, de- 
spite having important influence on adoption, point to the general conclusion that 
external inputs get used by the local schools in ways consistent with local needs. 

12 See Vol. II, Tables 11 and 17. This analysis acknowledges the fact that we dealt only with the 
explicitly micro-level outcomes of individual projects. Federal policies also have macro and oflen implicit 
concerns, such as fostering the legitimacy of bilingual instruction in public education, or the broad goal 
of Right-To-Read— increasing general awareness of the importance of reading. 

13 Because the change agent policy common to the programs had limited influence, each separate 
program could affect project implementation only marginally. Within the latitude possible for marginal 
effects, some significant differences between programs could be discerned. Appendix C presents our 
findings about these marginal differences. OCT 
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Thus, the basic policy instrument of differential funding had little if any influence 
on the motivations that led districts to initiate projects, that is, opportunism or 
problem solving In exchange for the federal grant, the districts essentially promised 
only to adopt the projects. The money in and of itself did not stimulate support, 
commitment, or interest in change. For example, projects characterized by problem- 
solving attitudes typicallj involved innovations that the district was committed to 
anyway. In contrast, projects based on opportunism generally lacked serious interest 
in change regardless of the extra money. Thus, since special funding'did not alter 
basic motivations, it failed to influence those features of initiation that shaped 

implementation and project outcomes. 
* * * 

In summary, our data show that a receptive institutional setting is a' necessary 
but not sufficient condition for effective implementation. An implementation strate- 
gy that promotes mutual adaptation is critical. 

The main factors affecting innovations were the institutional setting, particu- 
larly organizational climate and the motivations of participants, the implementa- 
tion strategy employed by local innovators to install the project treatment, and the 
scope of change implied by the project relative to its setting. Neither the technology 
nor the project resources nor the different federal management strategies influenced 
outcomes in major ways. Thus, projeqt>«outcomes did not depend primarily on "in- 
puts" from outside but on internal facfors and local decisions. 
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V. TENTATIVE POLICY IMPLICATIONS 



At this preliminary stage of the Rand research, we cannot propose detailed or 
firm policy recommendations. However, we can offer some building blocks for future 
policy that reflect our findings about school district behavior. 

>. . 

FEDERAL CHANGE AGENT PREMISES VERSUS LEA PRACTICES 

Our evidence indicates that federal change agent policies had their primary 
effect on the initiation of projects but that neither those policies that were unique 
to each of the federal programs nor those policies that were common to them had 
a strong influence on- the implementation of local innovations. Federal change agent 
policies exercised limited leverage on the course of innovations because they did not 
critically influence those factors most responsible for effective implementation— the 
motivations of actors within the institutional setting and the locally designed im- 
plementation strategies. These observations suggest that a number of premises 
underlying the formulation of change agent policies need to be reexamined. 

Federal policy appears to assume that school districts take something akin to 
an R&D approach to change, ,an assumption expressed in three main premises 
forming the foundation of change agent policies: 

1. The adoption by the schools of better technologies or treatments would 
lead to more effective educational practices. ^ 

2. The schools are motivated to search for better technologies. If they had 
reliable information that would make them aware of better practices, they 
would be willing to adopt them. 

3. The schools lack money to experiment with innovations. By providing 
"seed money," federal funds would allow school districts to try out new 
practices and to continue them if they prove to be successful. 

Our observations suggest, on the contrary, that while technology, information, 
and money are necessary to support innovations, they often play a different role in 
the initiation of innovations than federal planners expected. For example: 

1 - • The mere adoption of a "better" practice does not automatically or invaria- 
\ - ' bly fulfill its promise of "better" student outcomes. The reason, as our study 
shows, is that project implementation, which is largely shaped by the 
institutional setting, dominates the innovative process and its outcomes. 
This means that initially similar technologies installed in different settings 
undergo unique alteration and thus their outcomes cannot be predicted on 
the basis of treatment alone. 

2. The school districts seldom "searched" for better treatments; nor did infor- 
mation about promising practices seem to stimulate them to look outside 
of their districts. Instead, local planners tended to rely on technologies that 
were known and supported locally. When demand for change existed, it 
arose within the district in response to local needs, not generally as a result 
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of an awareness of purportedly successful innovations that took place else- 
where. 

3. Federal change agent funds did not appear to induce school districts to 
experiment or to take risks with significant innovations.. Instead, districts 
took advantage of the availability of these funds to support temporary 
. add-ons or to finance practices for which prior LEA commitments to solve 
a local problem existed. 



ALTERNATIVE CHANGE AGENT PREMISES AND POLICIES 

We believe that federal change agent policy could be based on premises about 
how'school districts behave that do not subscribe to an R&D model. Our first-year 
research suggests that the following premises better express the realities of LEA 
behavior in the innovative process: 

1. Implementation— rather than the adoption of a technology, the availabil- 
ity of information about it, or the level of funds committed to it— dominates 
the innovative process and its outcomes. 

2. Effective implementation depends on the receptivity of the institutional 
setting to change. 

3. Effective implementation is characterized by the process of mutual adap- 
tation. ». 

4. Local school systems?^ary in their capacity to deal with innovations and 
with the stages of the innovative process. 

The policy implications of these four premises are: 

- 1. Policy should be concerned with more than the mere adoption of change 
ag^nt projects. Federal change agent policy clearly stimulated the initiation of 
special projects, but had little effect on the quality or seriousness of the implementa- 
tion efforts. 

2. The critical significance of the institutional setting should come as no sup 
prise to policymakers. Indeed, the external inputs of educational technology, infor- 
mation, and money are intended to overcome local resistance to outside influence 
on internal decisions. Our research indicates, however, that they rarely do. If educa- 
tional technologies are not altered and adapted to local conditions, they are ineffec- 
tive; information about practices elsewhere seldom goes beyond the level of simple 
awareness; federal money is used for its intended purpose only if the federal purpose 
is congruent with local plans. 

School districts use the external inputs, but typically are not influenced by them 
to change th,eir commitments, motivations, or concern with innovation; unless the 
institution is receptive to change, it is unlikely to be stimulated by these policy 
instruments. We believe that policies could be designed to enhance receptivity to 
change. 

3. If, given a receptive institutional setting, a project's outcomes depend criti- 
cally on local decisions about huw the project will be implemented, federal policy- 
makers might consider ways of encouraging mutual adaptation strategies which we 
believe are the key to effective implementation. 
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It would not be easy to develop and carry out policies to encourage iputual 
adaptation. We found that attempts to provide "outside aid or prescribe management 
techniques were usually counterproductive, leading to nonimplementation or coop- 
tation. On one hand, outside experts were typically ignored as being too abstractor 
too unaware of local problems. On the other hand, focal staffs sometimes relied so 
heavily on technical assistance that they wes^unable to adapt project materials and 
methods to their own needs and thus they were unable to learn by doing. Similarly, 
the packaged management approaclies we observed tended either to be dismissed as 
unworkable or to be viewed as complete road maps to innovation, in which case 
project participants failed to develop the flexibility to cope with unanticipated prob- 
lems. , ■ ' 
' Such efforts to help districts innovate seem to be motivated, in part, fry interest, 
in fiscal efficiency and in accelerating the innovative process. Outsid^ assistance , 
may produce savings under certain conditions, but our research suggests that inno- 
vation is by its nature a costly and time-consuming process. Attempts to speed up 
th^process or to reduce its costs may achieve short-term efficiency at the expense 
of long-term benefits. For example, we observed cases iji which districts hoped to 
shorten the start-up period and' save money by using intensive, "one-shot^ pre- 
service training sessions. Such efforts were largely wasted because the inadequacy 
of staff members' experience with the project's precepts and problems prevented 
them from making effective use of the training. 

We belieye, however, that federal policymakers could formulate administrative 
guidelines that might support local innovators and state facilitators in the develop- 
ment of an adaptive implementation strategy. Such- guidelines could articulate the 
value of those elements that we found essential to mutual adaptation: 

• Continuous and on-line planning 

<• Regular and frequent staff meetings * < < 

• In-service training linked to staff meetings 

• Local material development - >, 

If these elements are criticize'c} as being time consuming or "inefficient/' it is impor- 
tant to remember that the pace of the change processes necessarily slow because 
people learn by doing. * * 

Federal policy also might assist "learning by doing" by using the present net- , 
work of regional labs in a revised role. For example, we argued that local material 
development promoted mutual adaptation; that is, "reinventing the wheel" enables 
school staff members to adapt project materials to their own needs— an essential 
step in fostering their commitment to the project. It may be.possible for regional labs 
to shift emphasis away from the preparation of complete packages of curriculum 
materials to "skeletal" frameworks that allow for and indeed assume development 
by local staff. In addition, the labs might evolve toward providing practitioner-based 
on-line assistance to districts that are implementing such skeletal frameworks. 1 

4. Fedeial change agent programs generally awarded fixed-term grants re- 
gardless of the school districts' ability to introduce and sustain the particular inno- 
vations represented in their proposals: Yet we observed similar innovations being 



1 The Northwest ltf|>ionul L.ih appears to have ewilved toward offering practitioner-based 
assistance * 
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approached and installed very differently by school districts according to their ca- 
pacities to innovate. For example, one district attempting to initiate a remedial 
mathematics project based on Piagetian principles was struggling to operationalize 
the philosophical principles of the project; another district funded at the same time 
and at an equivalent level was dealing with the project at a higher developmental 
level — it had refined the philosophical principles to meet, local needs and was in the 
process of producing classroom materials. When the three-year federal grant ran 
out, the project participants in the first district had learned to deal with the innova- 
tion but the district could not afford to continue it; the other district had used the 
federal money to begin districtwide dissemination. 

These situations are but two of many cases that suggest a possible new depar- 
ture for federal policy. Rather than blanket awards of a fixed number of years, 
federal change agent policies might be keyed to the stages of innovation and might 
promote the development of the school districts' capacity to deal with each stage. 

Although this interim report qannot recommend operational policies or proce- 
dures, we can explore some of the possibilities and the problems raised by the 
concept of a change agent policy keyed to the innovative stages: initiation, im- 
plementation, and incorporation. 

Initiation. Federal policy toward the initiation stage might aim to reduce op- 
portunism and, more important, aim to increase the school districts' receptivity to 
change by helping them create a local "demand" for innovation and develop a 
problem-solving attitude in understanding and coping with their needs. Our evi- 
dence suggests that the most effective federal or state incentives for these purposes 
may be indirect. 

For example, our research indicates that would-be innovators — teachers .or ad- 
ministrators — can more fully appreciate and better cope with the traumas and 
complexities of change if they have concrete, practical experience. Gaining such 
expedience within one's own limited sphere of activity in a district can be a costly 
and uncertain process. A more effective means may be provided by such devices as 
internship or training grants for administrators and travel money and release time 
for teachers to participate in innovative practices in other districts./These concrete 
experiences may enable administrators and teachers to expand their horizons, to 
learn what can be done by people committed to change, and to generate enthusiasm 
for innovation. We are suggesting ah opportunity for professionals to interact with 
their peers in work situations, rather than a "lighthouse" dissemination strategy for 
spreading an educational treatment. 2 Because local districts seldom seem able or 
willing to expend their limited resources for such purposes, competitive awards for 
professional growth might be federally financed. Such federal awards would carry 
prestige and thus reinforce the incentives for innovating. 

Federal policy also might offer proposal formulation grants to promote the ca- 
pacity of districts to plan adaptively, that is, to plan iteratively throughout im- 
plementation so as to make the continuing adjustments characteristic of mutual 
adaptation. Our evidence suggests that either too little or overly rationalistic plan- 
ning resulted in ineffective implementation. Instead, effective project planning be- 
gan during proposal formulation to lay an organizational groundwork for adapta- % 



2 We are not referring to short trips to educational fairs or professional meetings which d^olay 
innovations in an artificial setting. 40 
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tion by mobilizing the district leadership and support necessary for implementation, 
by increasing the flow of communication and feedback among staff and administra- 
tors, and by-involving project participants so they developed a sense of ownership 
toward the project. The federal award of a proposal formulation grant might be 
contingent on the school districts' stipulation about how they would organize them- 
sel t.? to plan adaptively to initiate a proposed innovation. 

Implementation. Before a project goes through its implementation stage, both 
fi j r^ : -c- agencies and local innovators experience considerable uncertainty about 
the risks and benefits of a proposed innovation— uncertainty that can only be re- 
solved by a trial implementation. By making funds available in such a way as to 
reinforce the trial aspect of implementation, federal policy might encourage schools 
to experiment and take risks— behavior that we found rarely occurred. 

Because errors—and indeediailures— cap be expected in the trial implementa- 
tion phase, federal planners should adjust their expectations to account for the e 
risks, especially in the area of evaluation. We found that summative evaluation 
oflen seemed to be ignored or treated as a necessary ritual. Rather than summative 
evaluation, federal policy might foster formative evaluation, which might not only 
remove the onus of failure (and thus reduce the disincentives that lead to cooptation 
and nonimplementation) but also prpvide essential feedback to local actors when it 
is most important (thereby fostering mutual adaptation). Accordingly, federal or 
state funds for trial implementation might be viewed as short-term (2 years) risk 
.:/pital and might be so publicised. An award mechanism similar tc the one used in 
the present Title III might be appropriate, perhaps with the requirement of a 
prerequisite propose! formulation grant. 3 

Incorporation. Because financial considerations often limit a school district's 
ability to incorporate and *v ead an innovation, federal policy might be able to 
influence incorporation directly. Incorporation grants of a relatively large amount 
might be established that would be awarded within categories that were determined 
by federal priorities. 

The use of such incorporation grants raises serious problems of equity and 
evaluation. These grants might be awarded to districts after they had received 
proposal formulation and trial implementation grants and been judged "successful" 
at each stage. This system of successive grants would filter out many projects from 
federal grant con^deration. Moreover, assuming that districtwide incorporation 
would be relatively costly in many cases, fewer projects might be funded at the final 
stage. In short, this particular sequential scheme of keying federal change agent 
policies to the stages of innovation implies that fewer but "b|tt6r" projects might 
be funded. Thus, it may sacrifice equity to gain quality, at least in the short run. 
Equity sus quality is an issue of value and politics that this research cannot 
resolve. 

Filtering projects for an incorporation grant would require thorough evalua- 
tion. Wje doubt that the usur.I selection device of judging proposals at a distance 
woald be effective. On-site visits might be necessary to evaluate such crucial ele- 

3 It would be naive to suppose that opportunity-basea* projects would not .be funded in this phase. 
Neither the federal government nor state government* have low-cost means available for distinguishing 
opportunistic projects from problem-solving t Mtcmp& Proposals usually sort along a dimension of skill 
in grantsmanship. and thus big or wealthy districts benefit. Although there is no sure q ick or easy 
solution to this problem, a required proposal formulation stage as p-eviousty discussed might filter out 
the more opportunity-based proposals ■ £L\ 
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merits as the local demand, the institutional capacity to innovate, and the district's 
problem-solving attitude. 4 Ofthe project-related criteria, our research indicates that 
the scope of the proposed change— i.e., the projects complexity, centrality, conso- 
nance, and the nature and amount of change required— and the implementation 
strategy would be critical areas to evaluate. 

In sum, there appear to.be many possibilities for federal policy to affect the 
innovation process despite its essentially local nature and the autonomy of school 
districts. Each possibilitv for federal leverage raises problems. This interim report 
has offered our preliminary thoughts about new policy directions. But, more impor- 
tant, it has tried to provide S' e information and hypotheses that would help 
policymakers balance the possib.iities and the problems that arise from federal 
efforts to help schools change themselves. 

4 The use of matching grants might be one way to assure the district's commitment and its belief in 
the centrality of the project 
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Appendix A 

RESEARCH DESIGN OF CHANGE AGENT STUDY 



This appendix presents a brief description of the tasks for the change agent 
study, as listed in Table A-l. The first sixteen tasks have been completed; Tasks 17 
through 20 constitute research plans designed to study (1) what happens to innova- 
tive projects after federal funding has ended and (2) how and under what conditions 
innovative projects spread to other sites within or outside of districts.. 



Table A-l 

Outline of Tasks for Change Agent Study 



Task 




Number 


PHASE I 


1 


Project summary 


o 


Literature review 


3 


Program analysis 


4 


Sample selection 


5 


Design and pretest questionnaires 


6 


Prepare study design; update summary report 


PHASE II 


7 


Conduct baseline survey 


8 


Preliminary data analysis 


9 


Subsample selection of projects for fieldwork 


10 


Intensive fieldwork 


11 


Telephone survey of SEA officials 


12 


Fieldwork at SEAs 


13 


Revise study design 


14 


Data analysis 


15 


Refine data collection plan for second year 


16 


Final reporU and executive summary, first year 


PHASE III 


17 


Design, develop, and pretest survey questionnaire 


17a 


Design telephone sereener questionnaire 


" 18 


Conduct second-year survey questionnaire 


18a 


Data analysis, second year 


19 


Conduct second-year fieldwork 


19a 


Comparative site visits 


20 


Final report and executive summary, second year 



o 



PHASE I 



Task 1. Project Summary 

During July 1973, we prepared an initial planning report outlining project 
purpose, tasks, and research approach as agreed to by Rand and USOE. 

43 
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Task 2. Literature Review 

We reviewed the literature on educational innovation, organizational change, 
and dissemination and developed working hypotheses about these processes in order 
to guide the design.of data collection and analysis. This review and the conceptual 
model are reported in Vol. L 

Task 3. Program Analysis 

We reviewed USOE program files and other pertinent literature, interviewed . 
USOE program officers, prepared a tyrology of program variable! of each of the 
selected change agen£ programs, and debcribed major policy issues connected with 
each change agent* program. 

Task 4. Sample Selection 

We selected a sample of change agent projects based on a stratification using 
key demographic and project characteristic variables. Details of the sampling proce- 
dure and characteristics of the survey sample obtained are reported in Vol II, App. 
A. 

Task 5. Design and Pretest Questionnaires * 

We developed and pretested questionnaires for the baseline survey. This task 
was conducted during the period July 15 to October 15, 1973. We developed separate 
questionnaires for each of the five classes of respondents with supplementary ques- 
tionnaire elements provided for each of four federal change agent programs being 
studied. Volume II> App. A, describes the i-ontents of the survey questionnaires. 

Task 6. Prepare Study Design; Update Summary Report 

We prepared a final study design detailing plans for the management of data 
collection, processing, and reporting functions, and also specified plans for data 
analysis. 

PHASE II 

Task 7. Conduct Baseline Survey A 

A baseline survey using structured questionnaires was conducted by the Na- 
tional Opinion Research Center during the period December 1973 and January 1974 
with 1735 respondents (293 projects in 18 states) at the local level. 

Task 8. Preliminary Data Analysis 

We used this preliminary data analysis to provide information for fieldwork. 

Task 9, Subsample Selection of Projects for Fieldwork 

On the basis of preliminary data analysis, we selected 29 sites and contacted for 
fieldwork. 44 
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Task 10. Intensive Fieldwork 

During April and May 1974, we conducted intensive fieldwork on c* nge agent 
projects in 29 selected school districts. In each district a two-person Rar ceam spent 
several days interviewing district and school-level officials and col' -ing data on 
change agent projects ih the cfistrict. The fieldwork focused on five ' .ds of projects: 
career education, bilingual education, -eading programs, staff development, and 
classroom organization projects. The results of each field trip were written up as case 
studies and an analysis of each case study area appears in Vol. Ill, Apps. A through 
D. Rand's approach to»lhe fieldwork is presented in Vol. Ill, Sec. I. 

Task 11* Telephone Survey of State Education Agency Officials 

We contacted 54 state officials during January and February 1974 in the 18 
states of the survey sample to elicit information about project descriptions and 
evaluations and about SEA management styles. The results of this survey were 
incorporated in each" of the Appendixes to Vol. III. 

Task 12. Fieldwork at State Education Agencies 

We visited 9 state education agencies to elicit information on state policies on 
project selection, technical assistance, dissemination, and priorities for educational 
change. The results of these visits were incorporated in each of the Appendixes to 
Vol. III. 

Task 13. Revise Study Design 

On the basis of preliminary analysis of survey and fieldwork, we revised the 
study design. 

Task 14. Data Analysis 

We analyzed data collected from survey and fieldwork, using multivariate sta- 
tistical procedures. The results are reported in Vol. II. 

Task 15. Refine Data Collection Plan for Second Year 

We prepared a detailed plan for collecting, processing, analyzing, and reporting 
data pertaining to Phase III, the Title III and Title VII continuation study. 

Task 16. Final Reports and Executive Summary, First Year 

We prepared final reports and an executive summary that presented empirical 
analyses. These analyses were reviewed and synthesized in Vol. IV, which also offers 
tentative policy implications. 



45 



34 



PHASE III 

Task 17. Design, Develop, and Pretest Survey Questionnaire 

We will develop and pretest questionnaires designed particularly to find out 
about the continuation and dissemination- of innovations that had been supported 
under Title III and Title VII. Survey questionnaires will be designed to be adminis- 
tered to teachers, principals, project directors, and superintendents. The survey 
sample will consist of Title III and Title VII projects that were in their last year of 
federal change agent funding during the 1973-74 school year or the 1974-75 school 
year and that were in Rand's first-year sample. The eligible pool of projects is 155 
Title III projects and 33 Title VII projects. From this pool, approximately 100 pro- 
jects will be selected for inclusion in the survey. The design and pretesting of the 
survey will begin June 1, 1975, and the questionnaires will be completed by Septem- 
ber 15, 1975. 

Task 17a. Design Telephone Screener Questionnaire 

We wiP. design and develop a telephone questionnaire of administrative officials 
of post-federal funding Title III and Title VII projects. This instrument will elicit 
preliminary information on continuation that will be used for fieldwork and site 
selections. The telephoning will begin in March 1975 for the first cohort and in 
September 1975 for the second cohort, if necessary., ; 

Task 18. Conduct Second-Year Survey Questionnaire 

We will conduct a survey at the local level of participants in Title III and Title 
VII projects whose federal funding has'ended. Approximately 800 teachers, 200 
principals, 100 project directors, and 100 superintendents will be respondents for 
mostly close-ended questionnaires in approximately 100 districts. The survey will be 
administered during November and December 1975. Rand will subcontract the 
survey administration work, as it did for the first-year survey for which NORC was 
the subcontractor. 

Task 18a. Data Analysis, Second Year 

We will analyze data collected from telephone and survey questionnaires using 
first-year data as a baseline. Data analysis will be conducted throughout most of the 
contract period with several periods of intense activity. During May and June 1975, 
the first year's data base will be analyzed to focus on the continuation question and 
to prepare information for mail questionnaire design and processing. After receipt 
of responses from the telephone questionnaire in April and September 1975, these 
new data will be analyzed to gain a comprehensive view of continuation in the Title 
III and Title VII sample and to prepare information for site selection for fieldwork 
and the survey. From January to April 1976, the data from the survey will be 
processed and analyzed using the data and the findings from tljeJirst year as a point 
of comparison so that changes in project focus, methods, activities, and behavior may 
be assessed. 46 
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Task 19. Conduct Second-Year Ficldwork 

Teams of Rand staff will conduct field visits to Title III and Title VII projects 
no longer receiving federal funds. These visits will include six Title III sites and six 
Title VII sites. Four sites will be visited in depth during the spring of 1975; eight 
sites w?tl be visited more briefly during the spring and fall of 1975. The emphasis 
of the early fieldwork will be on developing hypotheses about continuation and 
intra-district dissemination and particularly the role of the district in these issues. 
These hypotheses will serve as the basis for questionnaire design. The later field- 
work will be used for survey validation purposes. Insofar as possible, the sites visited 
will be those in which the first year's fieldworjt took place. 

Task 19a. Comparative, Site Visits 

We will visit nonsample sites that use USOE's Project Information Packages 
(PIPs) and/or NIE-sponsored Wisconsin Individually Guided Education (IGE). The 
purpose of this task is to observe (at approximately two sites) these two classes of 
innovations, which ^re of concern to federal policymakers, in order to compare them 
with other innovations in the Rand sample. Such comparisons will provide the Rand 
staff with a more informed basis for reaching policy conclusions at the end of the 
study. 

Task 20. Final Report and Executive Summary, Second Year 

We will prepare a final report and executive summary which addresses in par- 
ticular the question of how federal policy can help local districts continue innova- 
tions. The completion date for the final report and executive summary is September 
15, 1976. 
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Appendix B 



CHANGE AGENT PROGRAM GUIDELINES 
AND MANAGEMENT STRATEGIES 



This appendix sketches relevant elements of the federal change agent program 
guidelines and management strategies. Since the role of the state education agency 
(SEA) in the four programs introduces another possible source of variability, this 
appendix outlines the function and responsibilities of both the SEA and the USOE 
for each program. 



TITLE HI OF ESEA 

Title III was the first major USOE program to provide funds to local school 
districts for innovation. Since the authorizing legislation places no restrictions on 
educational areas in which projects can be funded, Title III provides a broad pro- 
gram of support for local innovation. 

The goals of Title III are to stimulate and assist in the development of model 
elementary and secondary school programs through grants to local districts, and to 
.support the spread of these models to other schools. Grants are awarded on the basis 
of competitive proposals submitted by LEAs. Title III is also intended to give school 
districts experience in managing innovation and encourage them to undertake local* 
ly funded efforts to innovate. 

When first authorized in April 1965, Title III was a Commissioner's program; 
funds were managed by USOE and went directly from USOE to local districts. Two 
years later, Congress amended Title III to give the states responsibility for manage- 
ment of Title III funds. 

The State Plan Program 

The 1967 amendments to Title III underwrote a great expansion in SEA staff 
. and in the Title III program as a whole. SEA staff working on Title III have increased 
from the 50 full- and part-time persons working in 1967 to over 450 full-time and 
483 part-time people today. 

The 1967 amendments also required that states set forth a plan to be reviewed 
and approved by the Federal Title III officials for spending their allotment of Title 
III funds. The federal office was empowered to withhold up to 50 percent of a state's 
allotment until a satisfactory plan was submitted. 

The first official guidelines for the State Plan program were issued in the fall 
of 1971. These guidelines specified criteria for the states to follow in designing a Title 
III management plan that would be evaluated by the federal State Plans Branch. 
The guidelines outline sever* main components for which the SEA was responsible:* 

• Educational needs assessment 

• Project development Aft 
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• Selection and funding 6f model projects 

• Project and program evaluation 

• Validation 

• Dissemination 

• Adoption 

The State Plan strategy also included criteria for organizing Title III at the 
state level, notably the creation of State Advisory Councils and provision for profes- 
si /nal staff development. 

Section 306 

In 1970-1971, three years after the conversion of Title III to the State Plan 
program, Congress re-established a Commissioner's program in Title III by amend- 
ing the legislation to establish the Section 306 program. Beginning in FY 1971, the 
authorization provided that at least 15 percent of Title III funds be allotted USOE, 
with the remaining 85 percent allocated to the states. The Section 306 program has 
the same broad legislative intent as the original Title III program and has been 
similarly managed. 

Budget 

Congress has always appropriated more money for Title III than for other 
change agent programs. Local projects are typically funded for three years Title III 
appropriations have been about $150 million a year, which is about twice the current 
budget for the largest of the other change agent programs (Title VII, Bilingual 
program). The appropriations for the Title III budget reached a peak of almost $190 
million in 1968, after a rapid increase from $75 million in the first year. After 1968, 
Title III appropriations fell back and stabilized at a level of about $150 million until 
FY 1975, when they were reduced to $120 million. The current appropriation is 
about one-third of the authorized level. The Title III legislation allocates the funds 
among states by formula, specifying that states should receive $200,000, plus an 
• amount in proportion to the school-age and total population of the state. 1 



TITLE VII, BILINGUAL EDUCATION PROGRAM 

The Bilingual Education program was enacted in 1968 as an amendment to 
ESEA. The legislation recognizes "the special educational needs of the large num- 
bers of children of limited English-speaking ability in the United States," and au- 
thorizes bilingual projects in local districts to meet these needs. More than any other 
program in the change agent study, the Bilingual program aims at public acceptance 
of a fundamentally new concept in public education; as a result, it has been marked 
by turbulence and conflict. 

Guidelines ( 

USOE spent the first five years (19684973) formulating the social role and 
1 U.S. Congress. PL 89-10, Sec. 302(a) (2). 49 
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objectives of bilingual educatum. developing curricula for different languages and 
grade levels, providing internee training for teachers, devising new assessment 
techniques, and stimulating x\w SEAs 1 interest by involving them in program man- 
agement. Of all the programs* i n the change agent study, Title VII began with the 
fewest available resources and the least developed program strategy. 

Title VIITocuses on poor children between the ages of 3 and 18 and areas that 
need bilingual education programs The law, however, is vague about what educa- 
tional programs it envisions. In the first year of the Bilingual program, the federal 
program office sent grant announcements and guidelines to SEAs, which then for- 
warded them to districts that might qualify. Great leeway was given to LEAs so that 
they could focus on their own specific needs. LEAs could propose almost any educa- 
tion project for the target population as long as two languages were used in instruc- 
tion and the history and culture of the non-English language group were taught. 

The legislative language merely said that 'Title VII funds are available for 
exemplary pilot or demonstration projects." There was little in the early guidelines 
to indicate that applicants would be participating in a nationwide effort to develop 
models of bilingual education. In later years, four components — instructional pro- 
gram, curriculum, community involverhent, and staff development — were urged on 
projects. But the initial guidelines did not clearly focus or, structure the program. 
Approximately half of the bilingual projects in the Rand sample were funded under 
the 1968 guidelines. 

More recently, the Bilingual program office has developed a program manual 
that clarifies ttf?(Ieiinition of a bilingual education project and provides informal 
funding guidelines. After the first year, project and budget size could grow, but were 
limited to a vertical expansion in grade levels. For instance, if a program began with 
a kindergarten class in the first year, it could add a 1st grade class the second year, 
to allow for the development of a continuous program for the students. However, 
federal funds could not be used to expand the project horizontally, for instance, to 
two or more kindergarten classes in the second year. After the third year of vertical 
expansion, LEAs were expected to absorb the cost of the highest grade level par- 
ticipating. 

State Participation 

The states had no official role in managing Title VII during the period covered 
by this report. SEA involvement with Title VII projects varies directly with state 
interest in bilingual education. In states that have not needed to.focus on bilingual 
education, responsibility for Title VII programs is usually housed in some other 
bureau, such as Title I, migrant workers, foreign language coordinator, and so on. 
In states where bilingual education has been a major concern, a fully functioning 
Title VII staff is the rule. 2 These staffs have no formal direct authority over the 
funded LEAs, but they do exercise influence over Title VII programs by means of 
their connection with LEA officials and USOE staff. In western states, for example, 
Title VII is a higher priority for SEAs. Accordingly, they typically devote considera- 
ble staff time monitoring projects, making regular site visits, holding workshops, 
and the like. 



2 The information about the operation of SEAs was derived from informal interviews with state 
officials- EZfl 
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Budget 

The budget of the Bilingual Education program has risen steadily since its 
inception to its present level of $85 million in FY 1975, but did so only after a 
friendly Congress overruled administration requests to limit the program's growth. 
In the first year of the program, FY 1969, the program funded 76 projects in 21 
states, with the $7.5 million appropriation. Most of the projects were in California 
and Texas, and by far the largest number included Spanish-speaking children. Most 
projects had budgets exceeding $100,000 and, following the legislation, were spon- 
sored by a LEA, a group of LEAs, or a LEA in conjunction with a college or univer- 
sity. The regulations limited projects to a maximum of five years of funding. 

After 1969, the proportion of grants to California and Texas dropped dramati- 
cally as the program grew and tried to spread to new areas. Legislation and program 
policy have encouraged more language groups to' participate in the projects. 

VOCATIONAL EDUCATION, PART D 

The Vocational Education, Part D, program was enacted in 1968 as part of the 
comprehensive amendments to the Vocational Education Act of 1963. The goal of 
the Part D program is "to stimulate new ways of bridging tlie gap between school 
and earning a living for young people" through providing grants to local districts % 
for exemplary career education projects. Each project was expected to develop a plan 
that incorporated four elements: 

1. Broad occupational orientation in elementary and secondary schools to 
increase student awareness of the range of career options open to them. 

2. Work experience a n£ cooperative educational studies. 

3. Specific training in job entry skills. 

4. Intensive occupational guidance and counseling during the last years of 
school. 

These goals are quite similar to what was later embodied in the concept of career 
^education. 

State Participation 

Half of the Part D program funds are managed by the states and half by USOE- 
The federal program office wanted the federally managed projects to provide exam-* 
pies for state and local educators, to emulate or modify/using state or local funds. 

The federal portion of the Part D program encouraged proposals by an an- 
nouncement sent to the executive officers of the state boards CoY vocational educa- 
tion and the state directors of vocational education- They, in turn, publicized the 
program and solicited Rroposals from local educational groups. On request, the 
federal program office sent a prospective applicant a manual on how to prepare a 
proposal, the program regulations, a booklet on exemplary vocational education 
^programs, and a brief bibliography of previous research 'put out by the Educational 
Resources Information Center (ERIC). The manual and regulations were purposely 
vague to allow local groups to formulate their own ideas within the parameters set 
up by the policy paper: No planningjjrants were given, and applicantsVere required 
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to develop complete 36-month operating plans as their proposals. 3 DHEW Secretary 
Finch decided to bypass this process in the case of 20 projects for Model Cities 
programs in specified cities. In these 20 cases, the Model Cities program planners 
were essentially given a grant and asked to write an operational plan for it. 

Beyond requiring that SEAs use funds to support exemplary projects within the 
categorical purposes-of the legislation, Part D guidelines do not dictate state man- 
agement strategies, and the federal program office has no supervisory power over 
the SEA. Thus, SEA administration of Part D funds varied greatly. For example, 
SEAs had various strategies for selecting and funding their projects. A number of 
states funded the same projects selected by the federal program office. Some states 
elected to fund their own large projects, comparable in size to the federal Part D 
project. Other states funded three or four smaller projects. One state funded hun- 
dreds of mini : grants. 

In states where a number of proposals were received, different techniques were 
used to select those that would be funded. In one state, staff members selected 
projects that "they feel will be the best" without any formal attempt at competitive 
ranking. In another, a unit outside of the regular vocational education staff ranked 
each of the proposals, using a standard rating form. The vocational education staff 
then funded the projects in the order of ranking. In one state, the SEA staff exercised 
the initiative in getting projects started. Those LEAs that wanted Part D funds 
notified the SEA of their interest. The SEA staff then presented the LEAs it selected 
with the particular projects it wanted developed. 

'The degree of SEA management of local projects also varied considerably, ap- 
parently as a direct result of the interest that the SEA had in any given project. In 
states where only a few projects— all of particular interest to trie state— were actual- 
ly funded, project monitoring was likely to be intense. In states funding a large 
number of projects, there was only token monitoring with only one or two visits a 
year. 

SEA dissemination strategy also seemed to vary with its funding strategy. 
States that funded only a few carefully developed projects were likely to be interest- 
ed in developing high-quality curriculum packages or exemplary project models, 
which could be applied in other LEAs. However, states that attempted to fund many 
projects, in order to involve as many LEAs as possible, devoted less attention to 
developing exemplary packages, and spent most of their effort on simply promoting 
the career education cause. 

Budget 

The Part D program has been funded at a stable level of $16 million— far below 
its $75 million authorization level. The legislation requires that the Commissioner 
of Education allocate $200,000 to each state. In addition, he is required to allocate 
the remainder o£the appropriation to the states in proportion to the population in 
each state between the ages of 15 and 19. The Commissioner and each SEA then 
divide the sum allocated to each state in half and administer the halves independ* 
ently. 

3 Applicants sent proposals simultaneously to USOE and to the state board of education. The states 
had 60 days to reject any proposal sent to them. From the remaining proposals, the federal program office 
staff and 16 outside reviewers selected the grantees in each state. 

52 
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In FY 1970, the Part D program funded its first round of projects. The federally 
administered Part D program funded one project in each state for a maximum of 
three years with budgets generally between $100,000 and $150,000 per year. The 
federal contribution to the budgets remained constant in each of three years, and 
grantees were not required, to provide any matching funds. However, they were 
required to state in their proposals, how they expected to finance the projects ^fter 
the expiration of the federal grant. Because the level of funding has remained stable, 
the program initiated few other new projects until the expiration of the first-round 
projects in FY 1973. 



RIGHT-TO-READ 

The Right-To-Read program, under the authority of the Cooperative Research 
Act, has developed a demonstration strategy for improved reading practices in the 
schools. 4 Right-To-Read designed a prototype problem-solving approach as a model 
for local districts to use in changing their reading programs at the school level. 
Right-To-Read supports a number of projects in local districts to demonstrate the 
approach. These projects, the first activities funded by Right-To-Read, were the ones 
we investigated for the change agent study. 

Guidelines 

The Right-To-Read local implementation strategy, called the "School-Based 
Plan of Action," prescribes the kind of innovation that a school district is expected 
to undertake, a planning process, and organizational guidelines. The key elements 
of the Plan of Action are: 

• Project schools should implement some form of the diagnostic/prescriptive 
approach to reading based on teaching by objectives that allows flexibility 
in the actual choice of curriculum and instructional methods. 

. Project schools should attempt a total approach to reading improvement 
Rather than changing one or two components of the school reading pro- 
gram, the program advocates a whole series of interrelated changes, such 
aS introducing new instructional methods, new curriculum materials, par- 
ent involvement, a reading center, and specialized staff* 
^■r— In each school all teachers and students, whether or not they have severe 
reading problems, Should be involved. This is called the whole school con- 
cept. 

• In each school, the principal should be the project director and should be 
fully responsible for project decisionmaking and management. 

4 Two additional components of the Right-To-Read program were not included in the change agent 
study: 

1. Community-based projects. 

2. Right-To-Read works with the SEAs to coordinate existing state and federal reading improvement 
funds and to develop the SEAs* capacity for training local educators in methods of planning and 
implementing reading improvement programs. Right-To-Read provides the states with technical 
assistance, and a small grant of funds for the administration and conduct of training programs for 
local educators. £* O 
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• Each school should appoint a Unit Task Force consisting of a central office 
staff member, the principal, two teachers, two parents, and, optionally, the 
school librarian to plan the project and oversee implementation. 

. Each project school should decide on its own reading improvement pro- 
gram and plan,i£by following an 11-step planning process kit designed by 
Right-To-Read staff. This planning process begins with a needs assessment 
(which has also been laid out in a kit form) and includes steps to select 
project objectives, instructional materials, diagnostic instruments, instruc- 
tional components, personnel, in-service training, and the project budget. 

\ Each project should emphasize staff development by spending 85 percent 
of the total budget on in-service training and other training activities. 

In addition, Right-To-Read provided each project with technical assistance from 
technical assistanceuteamsaocat^iiat five sites 1 across the country. Members of these 
teams visited projecte^npp;al^o.help with planning, in-service training, and 
problem solving. Team^rabers were specifically trained in the 11-step planning 
process and were supposed to work closely with projects during this phase of activity. 

t* 

State Participation * & 

Although the states are involved in other components of the Right-To-Read 
effort, management of the Right-To-Read demonstration school projects bypasses the 
states entirely. 

Budget • ' 

The funding of the Right-To-Read program has remained stable at the relatwely 
low level of $12 million. A portion of these funds has been spent on projects in N>th 
local school districts and communities to generate model reading programs that will 
be useful demonstration sites for SEAs and LEAS developing their own reading 
improvement programs. Tl?e school district projects are of two types: school-based 
projects, which are three-year grants of approximately $40,000 per year to a single 
school in selected local districts; andjarge-city projects, whicfc are three year grants 
of $100,000 per year to groups of several schools in each of the 21 largest cities in 
the country. 
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^ Appendix C 

COMPARISON OF PROGRAM EFFECTS ON 
IMPLEMENTATION DIFFICULTIES 
AND PROJECT OUTCOMES 



Our evidence, as u.s' d in Sec. IV, indicates that the federal change agent 
programs h$d approximate ^qual and relatively secondary effects on iinplementa- r 
tion and continuation, because the policy common to these programs had limited 
influence on the critical internal factors that primarily determine how projects were 
implemented and incorporated. In short, each federal program could affect the, 
irse of the innovati ve process and its-outcomes only at the margin. Witlfin the 
utitude possible for marginal effects, the differences in management strategies of 
the programs appeared to be related to significant but statistically sn^all differences 
in the projects' difficulties during implementation and in their "outcomes." * 



COMPARISON OF PROGRAMS ON DIFFICULTY OF 
IMPLEMENTATION 

Our research found that major problems in implementation stemmed from two 
sources. First, innovation was often a disruptive process. Problems arose that reject- 
ed inexperience of project participants in planning for change and adapting to its 
demands. 1 Second, projects encountered difficulties in conforming to the selection * 
mechanisms, administrative guidelines, and substantive ^priorities of the federal 
programs. Comparing the relative importance of particular implementation prob- 
lems for the different change agent programs, we found the following patterns: 2 * 

State Title III 

State Title III projects (which are locally conceived and awarded on a highly 
competitive basis) avoided severe problems resulting from unclear or unfamiliar 
techniques, perhaps because their techniques were locally chosen By contrast, un- 
clear goals was a significant source of difficulty for teachers in State Title III projects. 
The initial goals of these projects tended to be ambitious or arftbiguous, The ambigui- 
ty may stem from the competitiveness of Title III awards, which may inadvertently 
encourage broad "grantsmanship" claims in proposal writing. The overly ambitious 

'.Our analysis grouped implementation problems into four types: (1) Those associated with the 
substance of the proji s goals or educatiunai methods— namely, goals not sufficiently defined (i e , lack 
of specificity of goals). t * mplicated or unclear techniques (which is a dimension of complexity), and teacher 
unfamtlianty with materials or methods (which is dependent on the relative newness of the innovation), 
(2) problems resulting from deficiencies of the prior plans for implementation— namely, inadequate space, 
equipment, or materials, unanticipated requirements, unrealistic goals or schedule; or teachers ahcady 
overloaded with other commitments, (3) problems arising in the given institutional setting— namely, 
leadership or management problems, faculty or sta AT resistance to the pi uject, and parental or community 
opposition, and (4) problems arising from reduction in funding Volume II, Table 9, presents a catalogue 
of implementation problems along with frequenc^oX/esponse 

2 See Vol. H, Table 10 J>i> 
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goals may also reflect the opportunity that Title III provides for districts to take a 
chance on highly innovative and complex projects. But unclear goals can later create 
confusion and conflict. Title III gives school districts great freedom of action, which 
can be both a help and a hindrance to successful innovation: It may reduce poten- 
tial implementation problems arising from the selection of unclear techniques, but 
it may. increase" future stress if goals are overly- ambitious or poorly defined. 
* m » 

Federal Title III, Sec. 306 

Although both state-administered and federally administered Title III pro- 
grams allow LEAs to formulate projects for a broad range of objectives, Federal Title 
III projects were more likely to involve large-scale administrative changes in school 
' organization, such as the introduction of planning, programming, and budgeting 
principles. 3 The Federal Title III program has been announced every year starting 
in 1970 by means of letters from USOE to superintendents that indicated the ap- 
proximate grant size (e.g., in 1971, the USOE announced that it planned to award 
grants averaging $150,000 for each of three years). -These guidelines may have 
encouraged projects of considerable scope, as well as projects favored by the superin- 
tendent's oifice; in short, they may have provided incentives for "top-down" innova- 
tions. 4 In any event, the problems resulting in significant implementation difficul- 
ties for Sec. 306 projects were those associated with top-down innovations— problems 
of complicated or unclear techniques and problems^of teadher overloau. 

Title VII 

Bilingual projects were the hardest to implement. Inadequate materials, teach- 
er overload, and unrealistic goals or schedules seemed to be the main obstacles. 
. These problems, particularly that of obtaining appropriate bilingual curriculum 
materials and staff, may stem from the relative newness of bilingual education. But 
theyalso may reflect the complex and highly ambitious changes that some bilingual 
projects attempted. 

Vocational Education, Part D 

The analysis showed that only two problems cited by teachers were significantly 
related to the difficulty of implementing career education projects— unclear tech- 
niques and parental or comir nity opposition. The fieldwork conducted for career 
education projects suggests th * these projects rarely enjoyed serious support from 
the host district.^Since little proposal development was required by the sponsoring 
federal program, many districts tended to treat career education projects as grants 
in aid. Moreover, many projects were initiated as a result of minority group demands 
(in the Model Cities cases) or the desires of * small group of professionals to increase 
career activities. Typically, neither situation was conducive to gaining district-level 
support for the resulting' projects. 

3 See Vol. II. Table 6. 

4 Conversely. State Title III projects ottered smaller grants and greater spreading of resources among 
school districts, allowing for the funding of more "bottom-up" projects of the type initiated by teachers 
in a single school. 

4 See Vol. III. App. D bt> 
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In this unstable situation, attempts to innovate may be vulnerable to dissension 
and underdeveloped techniques. Or, projects may rot work out in part because 
teachers lack incentives to try to make them succeed, This failure may have nothing 
to do with career educaticfh, but only with the situation in which such projects were 
initiated. There is an odd corollary to this vulnerability: When such problems were 
not encountered, project teachers did not report implementation difficulties simply 
because little change was demanded of them. 

Bight-To-Read ' ^ * 

In the 'Right-To-Read projects— but for no other programs— teacher un- 
famfliarity with materials was a serious source of difficulty. Right-To-Read demon- 
stration projects prescribe broad management strategies and strongly encourage 
diagnostic/prescriptive approaches to reading. The way LEAs dealt with the- 0 
guidelines may have determined how difficult it was to implement the projects If 
teachers objected to, failed to understand, or could not use the complex new reading 
curricula, instruments, and record-keeping devices, the project could become ex- 
tremely frustrating. 

Second, in Right-To-Read projects, lack of prior planning was quite strongly 
related to implementation difficulties. Unanticipated requirements, inadequate 
materials, and teacher overload were also sources of difficulty. This suggests that the 
rational planning model implicit in these projects may inhibit the flexibility neces- 
sary to deal with day-to-day problems. Finally, parental or community opposition 
reportedly could substantially impede implementation of Right-To-Read projects. 
Reading instruction is a central or basic educational mission of schools and, there- 
fore, a very salient issue for parents. If they disapprove of or misunderstand the 
Right-To-Read approach— perhaps interpreting it as remedial in intent— teachers 
may encounter stiff and continuing resistance to the project. 

* * * 

In summary, the characteristic difficulties associated with implementing differ- 
ent federal programs demonstrate that thv management strategies had subtle influ- 
ences beyond the initiation stage. Insofar as federal programs affect initial project 
design, their different selection mechanisms, guidelines, and expectations also cre- 
ate soecific barriers that local innovations must surmount." 

\ 

COMPARISON OF PROGRAM EFFECTS OF PROJECT "OUTCOMES" 

As Sec IV discussed, we ^irectly compared the implementation outcomes and 
expected continuation of projects funded by thedifferent federal programs. We did 
so first by simply computing ,the means of Sample projects for each of the various 
outcome measures or dependent variables for each/ederal program; then we re- 
peated .these comparisons statistically allowing for variation of the educational 
methods used by projects within each program; finally, we compared the mean effect 
,of programs controlling for other project and institutional characteristics. The ma- 
jor result was that programs differed only marginally from each other on implemen- 
tation outcomes and expected continuation. Secondary findings involve the signifi- 
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cant but statistically small or marginal differences in project effects between pro- 
grams. 

Several marginal differences between programs could be discerned by the sim- 
ple comparison of means of project outcomes for each program. One significant 
difference involved career education. Projects funded by Vocational Education, Part 
D had the highest average reported/'percentage of goals achieved" and'Title III the 
lowest, although the difference between the two'is barely statistically significant. 
Yet, career education projects were/on the average least likkfyjx) result in teacher 
change; moreover, they were reportedly the lfcast difficult to implement, particularly 
when compared with the program with the most difficult projects, Bilingual. These 
gross comparisons suggest that career education projects achieved the highest per- 
centage of their goals because they were not attempting major innovations. Most 
such projects simply added career enrichment materials to the curriculum. Perhaps 
this .explains why career education projects, even though they were the "most 
successful," were the least likely to be continued. 6 

In contrast, Title VII bilingual projects were viewed as the most difficult to 
implement and as being the least successful in achieving their goals. Nonetheless, 
they were at least as likely to be continued as the average of projects funded by the 
other programs. Our field studies suggest an explanation for this apparent disparity. 
Local constituencies had mobilized support or created demand for bilingual educa- 
tion and thus produced pressure for a local and state commitment to deal with their 
problems, no matter how difficult the projects may have been to implement effective- 

ly. 

Another marginal effect involved Right-To-Rea<j. Superintendents reported 
that Right-To-Read projects were more likely to be continued than career education 
projects. Both the survey analysis and the fieldwork suggest that projects with 
educational goals that are perceived as rpore central or important to district educa- 
tional concerns are more likely: to be continued. Thus, Right-To-Read projects con- 
cerned with the core task of reading may have^been more strongly supported by 
school districts than enrichment-type career education projects. 

These gross comparisons are averages that may mask project variations withir^ 
programs. Therefore, we also compared the effects of various educational treatments 
on project outcomes within the same federal programs and found once again that 
program differences account for little of the variation in project outcomes. 7 

However, the analysis does reveal interesting patterns for State Title III, which 
funds the most diverse range of projects. For State Title III, the use of classroom 
organization methods was positively related to teacher change, to difficulty of im- 
plementation, and to percentage of goajs achieved. Title III enrichment methods, in 
contrast, were negatively related to perceived success arrl to fidelity of implementa- 
tion, probably because Title III enrichment projects tended to be diffuse. Intensive 
traditional staffing on Title III projects was significantly related to increased difficul- 
ty of implementation, but also positively related to the amount of teacher change. 
Nonetheless, intensive traditional staffing did not increase the percentage of goals 

* See Vol II, Tables 17 and 20. 

7 The educational treatments were diverse across the Rand sample. To deal with this diversity, we 
developed analytical measures that reduced the many different educational methods employed by change 
agent proiects to five common factors. See Vol. II. Sec. II. 
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achieved on Title 111 projects. Schoolwide administrative changes seemed to increase 
difficulty of implementation but not to affect other outcome '. 

These findings can be interpreted by recalling the nature of the Title III pro- 
gram. Title III projects were locally conceived and competitively chosen; they^there-"^/ ♦ . I 
fore included a wide mixture of projects, ranging froiu straightforward enrichmem 
.activities tcpeomplex attempts to alte* classroom organization. Similarly, they in-" % 
eluded the diyersej^nge of motivation for initiating change agent projects, from 
opportunism to problem solving. The Title III problem-solving projects that empha- 
sized sustained efforts in classroom innovations were difficult to implement, but 
were more likely to be successful in terms of both teacher change and percentage 
of goals achieved. In contrast, the numerous Title III opportunity-based projects that 
engaged, for example, in simple enrichment did not appear to have produced signifi- 
cant change nor to have achieved their goals. The diversity in the scope of Title III 
projects appears to have been matched by uneven outcomes. In short, the Title III 
management strategy may have yielded high risks, but it also had high returns. 

School districts may also have viewed Title III in those terms. Districts were 
more likely to continue Title III projects with major changes in classroom organi- 
zation than other Title III projects, even though the classroom organization projects 
were harder to implement and perceived as no more successful, at least in the short 
run. This finding also suggests that classroom organization change projects usually 
had district support and were thus likely to be continued whether or not they were 
implemented successfully during the period of temporary federal funding. 



59 



ERIC 



